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Abstract 

The  usefulness  of  the  vehicle  out-of-commission  (VOC)  performance  standard 
currently  utilized  by  the  United  States  Air  Force  (USAF)  to  compare  vehicle  maintenance 
units  is  evaluated  in  this  paper.  Because  the  Department  of  Defense  is  facing  budget  and 
manpower  reductions,  the  importance  of  productivity,  efficiency,  and  effectiveness  in  daily 
operations  is  being  stressed.  Unfortunately,  the  VOC  performance  measure  does  not 
adequately  apply  these  concepts.  By  examining  the  concepts  of  productivity,  efficiency, 
and  effectiveness,  the  inadequacy  of  the  VOC  measure  is  highlighted.  In  addition  to 
comparing  VOC  to  these  concepts,  the  VOC  measure  is  compared  to  the  essential 
characteristics  of  an  effective  performance  measurement  system.  A  review  of  research 
into  performance  measurement  is  conducted,  with  emphasis  on  USAF  transportation 
squadrons,  to  examine  perceptions  about  the  VOC  measure.  Because  the  VOC  measure  is 
viewed  as  inadequate,  a  review  of  performance  measurement  indicators  throughout  the 
government  is  analyzed  to  determine  the  type  of  performance  measurement  system  that 
should  be  used  for  vehicle  maintenance  units.  The  use  of  linear  regression  is  advocated. 

This  research  also  identifies  the  seven  independent  fectors  perceived  by  transportation 
officers  as  impacting  the  performance  of  vehicle  maintenance  the  most.  These  factors 
include  training  levels  of  assigned  personnel,  parts  availability,  available  manpower, 
budget  available  for  vehicle  maintenance,  tool  and  equipment  availability,  age  of  vehicle 
fleet,  and  experience  (#  of  years)  of  assigned  personnel  and  should  be  included  in  a 


vm 


regression  model  to  accurately  establish  and  compare  the  performance  levels  of  vehicle 
maintenance  units. 


IX 


EVALUATION  OF  THE  VEHICLE  OUT-OF-COMMISSION  STANDARD  FOR  AIR 


FORCE  VEHICLE  MAINTENANCE  UNITS 


I.  Introduction 


Background 

In  a  profit  organization,  management  effectiveness  and  eflSciency  is  usually  based  on  a 
measure  of  profit  such  as  retum-on-investment  or  eamings-per-share.  In  nonprofit 
organizations,  like  the  Department  of  Defense  Of^OD),  the  goal  is  to  achieve  socially 
desired  nonfinancial  objectives  (Todd  and  Ramanathan,  1994;  123).  Usually  the  goal  is  to 
provide  services  (Keating  and  Keating,  1981 :40).  Because  there  is  no  profit  measurement, 
nonprofit  organizations  are  forced  to  use  hard-to-quantify  management  effectiveness 
indicators  (D'Angelo,  1992:1-2).  In  nonprofit  organizations,  there  is  no  single  criterion 
for  resource  allocation,  cost/benefit  analysis  is  difficult,  managerial  performance  is  hard  to 
cross-compare,  decision-making  is  centralized,  and  it  is  difficult  to  cross-compare 
subordinate  units  (Anthony  and  Herzlinger,  1980:40-41). 

When  a  decision  is  made  about  resource  allocation,  the  effect  on  the  “bottom  line” 
(profit)  provides  a  common  basis  for  evaluating  alternative  uses  of  resources.  In  nonprofit 
organizations,  the  objectives  of  the  organization  have  nothing  to  do  with  profit  and  thus  a 
comparable  evaluation  is  impossible  (D’Angelo,  1992:7).  There  are  many  different  types 
of  well  tested  cost/benefit  analysis  techniques  that  profit  organizations  can  use  when 
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comparing  different  proposals  in  determining  which  proposal  generates  the  most  profit. 
With  no  profit  measure  in  nonprofit  organizations,  it  is  hard  to  quantify  the  benefits  of 
competing  proposals.  Evaluating  managerial  performance  in  a  profit  organization  is  linked 
to  decisions  the  manager  makes  in  an  attempt  to  increase  profit,  and  this  profit  measure 
provides  a  common  thread  to  evaluate  and  compare  managers  throughout  the 
organization.  Managers  in  nonprofit  organizations  strive  to  stay  within  budget  limits  and 
sometimes  are  forced  to  accept  decreased  quality  and  customer  service  levels  in 
accomplishing  the  mission.  Decision-making  in  the  profit  organization  can  be 
decentralized  because  managers  are  aware  that  any  action  taken  should  support  the 
objective  of  profit.  Without  a  single,  all  encompassing  objective  such  as  profit,  decision¬ 
making  in  the  nonprofit  organization  is  often  centralized  to  ensure  the  diverse  goals  and 
objectives  of  the  organization  are  met  (D’Angelo,  1992:7). 

For  nonprofit  organizations  there  is  no  measure  of  output  that  compares  with  the 
effectiveness  of  profit  measures  available  in  profit-seeking  firms  (Keating  and  Keating, 
1981 :43).  This  inherent  nature  of  military  organizations,  where  management  decisions  are 
based  on  available  resources,  makes  productivity  difficult  to  measure  (Anthony  and 
Herzlinger,  1980:5).  Increased  economic  constraints  are  forcing  the  DOD  to  operate  with 
fttnptiasiR  on  the  effective  and  efficient  utilization  of  allocated  resources  to  produce  goods 
and  services,  and  inadequate  performance  measures  are  hindering  this  effort. 

As  Congress  continues  to  reduce  the  DOD's  budget  and  force  size,  the  challenge  to  the 
military  manager  is  to  do  more  with  less.  The  Air  Force's  obligation  authority  is  expected 
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to  grow  by  $500  million,  but  real  growth  will  shrink  by  2. 1  percent  in  fiscal  year  1995 
(Fulghum,  1994:24).  The  force  structure  will  be  cut  by  10-15  percent  in  fiscal  year  1995 
and  the  number  of  flying  wings  will  be  reduced  fi*om  20  to  17  by  1999  (Fulghum, 
1994a:25).  As  a  result,  Air  Force  managers  face  reduced  operating  budgets  and  decreased 
manning  levels.  To  make  efficient  and  effective  use  of  limited  resources,  managers  must 
have  adequate  performance  indicators  to  aid  in  the  decision-making  process.  Reductions 
in  the  Air  Force  are  not  unlike  those  currently  experienced  in  civilian  industry. 

Many  senior  executives  in  a  multitude  of  industries  have  had  to  rethink  how  they 
measure  performance.  For  example,  at  Edmonton  Telephone  Corporation,  increased 
competition,  new  technology,  and  increased  customer  service  levels  have  forced 
management  to  implement  new  performance  measurement  systems  (Meadows  and  others, 
1994:17).  Because  private  industry  and  the  Air  Force  have  recently  adopted  new 
management  approaches,  like  Total  Quality  Management,  both  organizations  have  realized 
that  if  you  change  management  strategy,  your  measurement  system  should  also  change  to 
be  consistent  with  this  new  strategy  (Kaplan,  1993:144). 

The  emphasis  on  effective  and  efficient  use  of  resources  was  highlighted  by  the  1993 
Government  Performance  and  Results  Act,  which  evaluated  strategic  planning  and 
performance  measurement  in  the  government.  This  act  emphasized  improving  federal 
program  effectiveness  by  focusing  on  results,  quality,  and  customer  satisfaction  (Duquette 
and  Stowe,  1993:29).  In  addition,  a  Majority  Staff,  House  Committee  report  on 
government  operations,  entitled  "Managing  the  Federal  Government  -  A  Decade  of 


3 


Decline,"  mandates  government  efficiency  and  effectiveness.  To  further  emphasize  the 
government's  commitment  in  this  area,  President  Clinton  submitted  a  report  to  Congress, 
entitled  “Vision  of  Change  for  America.”  This  report  calls  for  a  shift  to  what  he  calls 
"entrepreneurial  government"  intended  to  make  the  government  more  efficient  and 
effective  by  abandoning  practices  that  impede  flexibility  and  waste  resources  (Duquette 
and  Stowe,  1993:48). 

Effectiveness  and  efficiency  are  directly  related  to  productivity.  Mali  defines 
productivity  as  "the  measure  of  how  well  resources  are  brought  together  in  organizations 
and  utilized  for  accomplishing  a  set  of  results.  Productivity  is  reaching  the  highest  level  of 
performance  with  the  least  expenditure  of  resources."  He  also  states  that  "productivity  is 
a  combination  of  effectiveness  and  efficiency."  A  study  of  the  federal  government 
conducted  by  the  Civil  Service  Commission,  the  General  Accounting  Office,  and  the 
Office  of  Management  and  Budget  and  Audit  has  stressed  the  importance  of  this  definition 
and  its  applicability  to  government  organizations  (Mali,  1978:83).  Realizing  the 
importance  of  efficiency  as  early  as  1968,  the  Air  Force  began  developing  information 
management  systems  to  aid  with  measurement  of  various  transportation  functions. 

Transportation  Measurement 

In  1968,  the  Air  Force  established  a  computerized  information  system,  known  as  the 
Transportation  Integrated  Management  System  (TRIMS).  The  system  was  designed  to 
create  a  data  base  capable  of  interfacing  with  other  functional  areas  within  transportation 
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for  on-line  data  retrieval  (Directorate  of  Transportation,  1968: 1-l  - 1-5).  This  program 
was  never  fully  established  and  the  Vehicle  Integrated  Management  System  (VIMS),  a 
subsystem  of  TRIMS,  was  established  in  1970.  This  system  was  designed  to  provide  data 
to  improve  management  reports  and  enable  managers  to  resolve  vehicle  management 
problems  using  accurate  information  (Directorate  of  Transportation,  1969: 10-1 1).  In 
addition,  performance  indicators  were  established  as  part  of  VIMS  to  help  analyze  and 
compare  performance  levels  of  vehicle  maintenance  units.  The  primary  indicator  of 
performance  for  vehicle  maintenance  units  is  the  vehicle  out-of-commission  (VOC)  rate. 

It  is  a  combination  of  the  number  of  hours  a  vehicle  is  deadlined  for  maintenance  and 
parts.  Vehicles  deadlined  for-parts  (VDP)  are  those  that  are  placed  out  of  service  due  to 
nonavailability  of  parts.  Vehicles  deadlined  for-maintenance  (VDM)  are  vehicles  placed  in 
an  awaiting  maintenance  status.  Figure  1  graphically  portrays  the  relationship  between 
VDM,  VDP,  and  VOC.  VOC  is  the  overall  measure  of  support  given  to  its  customers 
(Department  of  the  Air  Force,  1994:3). 


Vehicle  Out-Of-Commission 

(VOC) 

[Vehicles  Deadlined 

Vehicles  Deadlined 

For  Parts 

For  Maintenance 

1  (VDP) 

Figure  1.  VOC  Components 
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According  to  AFM  77-3 10  Vol.  H,  the  VOC  standard  for  all  bases  is  10  percent 
(Department  of  the  Air  Force,  1987:12).  Table  1  shows  how  VOC  rates,  by  base,  are 
reported  to  HQ  AFMC  and  Figure  2  graphically  illustrates  VOC  by  base.  Although  Air 
Force  regulations  divide  VOC  rates  into  VDP  and  VDM,  AFMC  has  recognized  the 
importance  of  budget  levels  available  for  vehicle  maintenance  units  and  has  created 
another  category  called  vehicle  deadlined  for  funds  (VDF)  (Moon,  1995a). 


Table  1.  Components  of  the  VOC  Rate  (Moon,  1995) 


AFMC  BASES 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

VDM 

4 

10.4 

6.1 

■a 

4 

3.5 

3 

3 

■Q 

3.2 

VDP 

0.2 

2.6 

1.5 

1.9 

0.2 

1.4 

1.6 

2 

0.2 

0.8 

0.7 

VDF 

0 

0 

0 

0.7 

0 

0 

0 

0 

0 

0 

0 

VOC 

13 

wm 

6.5 

5 

3.2 

5.2 

3.9 

By  examining  this  performance  measurement  report,  managers  are  not  able  to 
determine  the  underlying  reasons  for  particular  VDM,  VDP,  or  VDF  figures.  As  a  result, 
the  VOC  standard  does  not  aid  the  decision-making  process.  Although  this  performance 
indicator  was  developed  over  twenty  years  ago,  it  is  still  considered  to  be  the  primary 
means  of  evaluating  performance  of  vehicle  maintenance  units  throughout  the  Air  Force. 
However,  this  indicator  does  not  account  for  differences  in  operating  factors  that  impact 
performance  levels  of  vehicle  maintenance  units  such  as  unit  manpower  levels,  age  of  the 
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vehicle  fleet,  size  and  age  of  maintenance  facility,  budget  level,  training  level  of  personnel, 
and  parts  availability. 


Figure  2.  VOC  Rate  Depicted  Graphically  (Moon,  1995). 


Status  of  VOC  Measurement 

Research  conducted  in  1989  by  ILT  Ke\dn  N.  Brewer  on  Perceptions  of  Air  Force 
Base-Level  Transportation  Officers  Towards  The  Effectiveness  of  Air  Force  Base-Level 
Transportation  Performance  Measurement  Indicators  showed  substantial  dissatisfaction 
with  current  measurement  indicators  among  transportation  officers  at  all  levels.  This 
research  pointed  out  that  independent  variables  such  as  manpower,  environment,  and 
budget  were  not  considered  when  measuring  performance  and  that,  as  a  result,  cross¬ 
comparison  of  units  should  not  be  performed  (Brewer,  1989:95). 


Comparison  of  units  using  VOC  measurements  is  not  accurate  and  Brewer  concluded  it 
should  not  be  used  to  cross-compare  units.  However,  it  is  used  habitually  by  the  Air 
Force  in  rating  the  performance  of  its  vehicle  maintenance  units.  Air  Force  managers 
should  not  make  the  same  mistake  made  with  numerous  other  management  tools  and 
assume  this  performance  measure  is  valid  for  all  situations.  Different  measures  should  be 
utilized  “according  to  the  reasons  for  which  they  were  developed”  (Flynn,  986:393). 

VOC  was  developed  to  measure  customer  satisfaction,  but  has  developed  into  a  global 
indicator  of  vehicle  maintenance  performance.  The  recent  adoption  of  the  Total  Quality 
(TQ)  philosophy  towards  management  within  the  Air  Force  resulted  in  a  change  in 
strategy  that  should  be  linked  with  performance  measures.  However,  the  VOC 
performance  measure  established  in  1970  does  not  lend  itself  to  the  TQ  philosophy  being 
used  today.  The  performance  measure  should  change  with  the  strategy  and  keeping  the 
same  measure  in  place  for  years  should  be  avoided  (Schmenner,  1993:524). 

Specific  Problem  Statement 

The  current  vehicle  out-of-commission  standard  is  not  indicative  of  how  a  vehicle 
maintenance  unit  is  performing  and  can  not  be  used  to  effectively  cross-compare  units. 

Research  Question 

Determining  the  best  measure  of  efficiency,  effectiveness  and  productivity  for  Air 
Force  vehicle  maintenance  units  that  will  allow  accurate  cross-comparison  of  these  units  is 
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the  main  goal  of  this  research.  The  research  question,  therefore,  is  “what  is  the  most 
appropriate  performance  measurement  system  for  Air  Force  vehicle  maintenance  units?” 


Investigative  Questions 


The  primaiy  investigative  questions  that  need  to  be  answered  are: 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 


How  are  efficiency,  effectiveness,  and  productivity  related  to  performance 
measurement? 

Why  are  more  relevant  performance  measurement  indicators  needed  in  government 
organizations? 

How  do  vehicle  maintenance  officers  perceive  the  VOC  performance  standard? 
Should  independent  factors  be  included  when  establishing  a  performance  standard? 
Is  multiple  regression  analysis  suitable  for  evaluating  the  performance  of  vehicle 
maintenance  units? 

What  independent  variables  impact  the  performance  of  vehicle  maintenance  units  the 
most? 

Are  there  differences  in  perceptions,  based  on  rank,  transportation  experience,  and 
current  position,  about  the  VOC  standard? 

Are  there  differences  in  perceptions,  based  on  rank,  transportation  experience,  and 
current  position,  about  the  most  important  independent  variables  impacting  vehicle 
maintenance  performance? 


Scope 

This  research  evaluates  the  vehicle  out-of-commission  standard  used  by  management 
to  analyze  and  compare  performance  of  vehicle  maintenance  units.  This  research  is  limited 
to  vehicle  m^tenance  units  in  the  USAF.  Independent  variables  that  impact  the 
performance  of  vehicle  maintenance  units  the  most  will  be  identified  by  transportation 
experts. 
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Summary  of  Ideas  Presented 

In  this  section  we  described  the  differences  in  determining  management  effectiveness  in 
profit  and  nonprofit  organizations.  We  also  highlighted  the  problems  facing  the  DOD  in 
terms  of  budget  and  manpower  reductions.  These  constraints  have  forced  government 
organizations  to  emphasize  the  importance  of  efficiency,  effectiveness,  and  productivity  in 
daily  operations.  Unfortunately,  current  performance  measures  of  vehicle  maintenance 
units  do  not  adequately  apply  for  these  concepts.  The  specific  problem  is  that  current 
vehicle  out-of-commission  standards  are  not  indicative  of  how  a  vehicle  maintenance  unit 
is  performing  and  can  not  be  used  to  effectively  cross-compare  units.  To  address  this 
problem,  this  research  will  attempt  to  determine  the  best  measure  of  efficiency, 
effectiveness  and  productivity  for  Air  Force  vehicle  maintenance  units  that  will  also  allow 
accurate  cross-comparison. 

The  literature  review  in  chapter  two  will  answer  investigative  questions  one,  two  and 
five.  The  relationships  between  performance  measurement  and  the  essential  characteristics 
of  productivity,  effectiveness,  and  efficiency  will  be  examined.  This  review  will  evaluate 
the  need  for  more  relevant  performance  measurement  indicators  in  Air  Force  organizations 
and  suggest  a  performance  measurement  system  that  could  be  used  by  vehicle  maintenance 
units. 
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n.  Literature  Review 


Introduction 

This  literature  review  examines  relevant  information  about  the  components  of  a  good 
performance  measurement  system.  Additionally,  the  concepts  of  productivity,  efficiency, 
and  effectiveness  and  how  they  are  related  to  performance  measurement  will  be  examined. 
Next,  a  review  of  previous  research  into  performance  measurement  will  be  conducted, 
with  emphasis  on  USAF  transportation  squadrons,  to  examine  perceptions  about  the  VOC 
standard.  Finally,  performance  measurement  indicators  throughout  the  government  will  be 
analyzed  to  determine  the  type  of  performance  measurement  system  that  should  be  used 
for  vehicle  maintenance  units. 

Performance  Measurement 

Performance  measurement  is  more  than  a  simple  measure  of  output.  The  primary  goal 
of  a  performance  measurement  system  is  to  create  meaningful  data  that  can  be  used  to  aid 
the  decision-making  process.  Other  important  goals  of  an  effective  performance 
measurement  system  include  improving  resource  allocation,  fostering  fact-based 
management,  and  providing  evidence  of  success  or  failure  (Ghobadian  and  Ashworth, 
1994:49).  Performance  measures  allow  managers  to  assess  outputs  in  relation  to  meeting 
organizational  objectives  (PoUitt,  1986:315). 
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Past  measurement  systems  provided  detailed  information  that  was  rearward  looking 
and  forced  the  manager  to  be  reactive  instead  of  proactive.  Such  is  the  case  with  the  VOC 
measure.  For  example,  a  rating  of  1 1%  reflects  the  number  of  vehicles  that  are  out  of 
service  at  a  given  period  of  time  or  for  a  given  period  of  time.  When  vehicle  maintenance 
officers  want  to  know  the  factors  contributing  to  this  rating,  additional  research  must  be 
conducted.  Determining  the  factors  underlying  this  rating  can  be  a  difficult  and  time 
consuming  task  and  on  many  occasions  there  is  not  enough  time  available  to  conduct  the 
research  to  aid  the  decision-making  process.  The  ideal  performance  measurement  system 
would  provide  information  pertaining  to  the  factors  causing  the  1 1%  VOC  rate  and  allow 
the  manager  to  make  timely  changes  to  correct  the  situation.  For  example,  a  lack  of 
training  could  be  a  cause  of  an  unacceptable  VOC  rate.  Scheduling  classes  that  provide 
instruction  on  how  to  repair  vehicles  quickly  and  efficiently  could  solve  this  problem.  For 
managers  to  act  in  this  proactive  manner,  the  performance  measurement  system  must  be 
able  to  provide  relevant  information  before  the  decision  has  to  be  made.  Therefore,  the 
system  must  be  designed  to  provide  pertinent  and  timely  information  (Ellis,  1993:16). 

Government  agencies  recognized  for  high  achievement  use  their  measurement  systems 
to  aid  in  decision-making.  These  types  of  measurement  systems  are  designed  to  answer 
the  following  questions:  Is  the  organization  performing  its  job?  Is  it  responsive?  Is  it 
productive  (Holzer  and  Callahan,  1993:33 1)?  Is  the  organization  achieving  a  higher 
throughput?  Is  the  organization  achieving  lower  operating  expenses  (Ronan  and  Pass, 


12 


1994:10)?  Answering  these  questions  involves  comparing  measurements  of  actual 
performance  against  a  standard  of  performance. 

Generally,  three  types  of  standards  are  applicable  for  this  comparison.  A 
predetermined  standard  can  be  established  by  reviewing  an  organization’s  goals  and 
objectives.  Historical  standards  are  based  upon  records  of  past  performance  and  can  be 
used  to  compare  performance  levels.  External  standards  are  used  to  compare  similar 
organizations  subject  to  the  same  operating  factors  that  affect  performance.  The  best 
measurements  compare  actual  performance  against  a  predetermined  target  of  performance 
(Thom,  1980:33).  If  actual  performance  is  lower  than  the  target  performance,  this 
difference  represents  the  lack  of  satisfactory  performance  towards  a  particular  objective. 
The  extent  to  which  the  actual  measurement  matches  the  target  will  define  the  measure  of 
productivity  (Sardana  and  Prem  Vrat,  1987:108).  Actual  VOC  rates  are  compared  to  a 
predetermined  target  of  performance  (10  percent)  intended  to  represent  customer 
satisfaction.  However,  no  evidence  exists  to  suggest  that  customers  had  input  in 
determining  the  target.  As  a  result,  the  target  could  be  an  invalid  representation  of 
customer  satisfaction. 

A  major  problem  with  the  selection  of  variables  used  in  performance  measurement 
systems  is  that  the  variables  do  not  measure  all  aspects  of  actual  inputs  and  outputs 
(Mentzer  and  Konrad,  1991 :36).  In  an  effort  to  overcome  this  problem,  all  factors  related 
to  the  final  output  of  a  product  or  service  should  be  considered  when  designing  a 
measurement  system  (Kearney,  1984:42).  Because  there  are  many  different  opinions 
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pertaining  to  the  characteristics  of  an  effective  performance  measurement  system, 
establisWng  such  a  system  is  a  difficult  task.  A.T.  Kearney  established  seven  criteria 
essential  to  an  effective  performance  measurement  system: 

1.  Validity.  The  most  valid  measure  is  one  that  accurately  exemplifies  changes  in 
productivity 

2.  Coverage.  The  more  completely  a  measure  covers  all  uses  of  a  resource,  the 
more  fully  the  resource  can  be  tracked. 

3 .  Comparability.  For  a  productive  measure  to  be  traced  over  time,  it  must 
contain  a  common  denominator  that  allows  for  the  comparison  of  other 
organizations  at  different  locations. 

4.  Completeness  This  emphasizes  the  extent  to  which  the  resources  used  to 
produce  a  service  or  good  are  accounted  for. 

5.  Usefulness.  The  measure  must  be  of  use  to  the  manager  in  making  decisions. 

6.  Compatibility.  The  measure  must  be  compatible  with  existing  data  retrieval  and 
reporting  methods  to  be  easily  implemented  in  an  organization. 

7.  Cost  Effectiveness.  The  ultimate  cost  of  the  measurement  system  must  be 
offset  by  the  savings  realized  through  the  use  of  the  system. 

(Kearney,  1984:42-43) 

Brizius  and  Campbell,  and  Hatry  and  others,  characterize  an  effective  measurement  system 
in  much  the  same  way: 

1.  It  should  be  focused  on  outcomes  and  quality— not  processes. 

2.  It  should  help  managers  improve  their  organization. 

3.  It  should  de&e  outcomes  and  quality  fi'om  the  customer’s  perspective. 

4.  It  should  use  a  few  select  indicators  for  managers. 

5.  The  information  produced  should  be  useful  to  policy  and  operational  decision 
makers  to  help  improve  their  operations. 

6.  The  data  should  be  consistent  and  valid  over  time. 

7.  Comparisons  should  be  provided  in  relation  to  standards,  baseline  data,  or 
targets  of  desired  performance. 

8.  Reports  should  be  made  available  to  managers  and  policy  makers  on  a  recurring 
basis  that  are  easy  to  read  and  understand.  (Kamensky,  1993:401) 

According  to  Vitale,  Mavrinac,  and  Hauser  the  optimal  measurement  system  should 

adhere  to  the  following  criteria: 
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1.  Accessibility.  The  measurement  system  should  use  data  that  is  already  available 
and  easy  to  retrieve. 

2.  Conceptual  simplicity.  The  strategic  and  operational  significance  of  the 
measures  should  be  easy  to  understand.  Any  measure  in  which  the  desired 
direction  of  movement  is  not  clear  should  be  avoided. 

3.  Relevance.  Measures  should  be  actionable  and  relate  to  the  process  or  output 
tracked.  If  performance  does  not  meet  standards,  managers  should  be  able  to 
recognize  the  source  of  the  problem  and  how  best  to  correct  the  problem.' 

4.  Reliability.  The  measure  should  track  true  performance  and  eliminate  “noise.” 

5.  Dynamism.  The  measure  should  be  flexible  and  capable  of  change. 

(Vitale,  Mavrinac,  and  Hauser,  1994:15) 

For  the  purpose  of  this  research,  the  essential  characteristics  of  a  performance 

measurement  system  must  include  the  following: 

1.  The  system  must  accurately  track  all  resources  utilized  in  the  process. 

2.  The  system  must  aid  decision  making  by  identifying  problem  areas. 

3.  The  system  must  use  data  that  is  compatible  with  existing  information  systems. 

4.  The  system  must  be  able  to  compare  performance  of  organizations  located  in 
different  locations. 

5.  The  system  must  track  true  performance  by  emphasizing  changes  in 
productivity. 

Another  guideline  to  follow  when  establishing  an  effective  performance  measurement 
system  is  limiting  the  number  of  measures  to  the  minimum  needed  to  achieve  the  desired 
results  and  ensure  they  are  related  to  the  objective  (Hendricks,  1994:27).  If  a  program 
has  more  than  15  measures,  they  should  be  reviewed  to  validate  their  importance  (Meyer, 
1994:96).  Additionally,  the  measures  should  be  easily  monitored  and  displayed  at  all  times 
(Wiley,  1994:362).  McMann  and  Nanni  stress  these  characteristics  by  stating  that 
measures  should  be  easily  understood,  simple  to  accomplish,  easy  to  manage,  and  cost 
effective  (McMaim  and  Nanni,  1994:56). 
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VOC  is  the  primary  performance  measurement  indicator  of  vehicle  maintenance  units 
throughout  the  Air  Force.  Vehicle  maintenance  performance  is  measured  in  terms  of 
achieving  a  10%  (or  less)  VOC  rate.  If  this  standard  is  achieved,  the  unit  is  considered  an 
adequate  performer.  Although  this  measure  uses  data  that  is  compatible  with  existing 
information  systems,  this  measure  does  not  possess  the  other  essential  characteristics  listed 
above.  VOC  does  not  track  the  resources  utilized  in  the  maintenance  process.  VOC 
tracks  the  time  between  the  opening  and  closing  of  a  work  order  without  any  regard  for 
resource  allocation.  This  measure  of  performance  also  lacks  the  ability  to  aid  decision 
makers.  Areas  of  poor  performance  are  not  emphasized  by  this  measure.  If  the  vehicle 
maintenance  officer  wants  to  determine  the  causes  of  the  VOC  rate,  extensive  research 
must  be  conducted.  Typically,  VOC  rates  are  used  to  compare  maintenance  units  from 
different  bases.  This  comparison  is  invalid  because  VOC  rates  do  not  accoimt  for 
differences  in  independent  factors  such  as  manpower,  budget  levels,  and  age  of  the  vehicle 
fleet.  Finally,  VOC  rates  do  not  track  true  performance.  A  unit  may  achieve  a  lower 
VOC  rating  at  the  expense  of  performing  quality  maintenance.  As  quality  maintenance 
decreases,  the  time  to  close  work  orders  also  decreases.  Although  the  VOC  rate  has  been 
reduced,  true  performance  of  the  unit  has  decreased. 

A  study  conducted  by  The  Maryland  Center  for  Productivity  and  Quality  of  Worklife  in 
1981  for  the  Air  Force,  reported  by  Howell  and  Van  Sickle  in  Perceptions  Of  A 
Methodology  For  The  Development  Of  Productivity  Indicators,  also  listed  three  broad 
guidelines  of  an  effective  performance  measurement  system.  First,  performance 
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measurement  systems  should  include  the  concepts  of  productivity,  efficiency  and 
effectiveness.  Second,  measures  of  efficiency  and  effectiveness  should  include  such 
characteristics  as  completeness,  comparability,  and  acceptability  to  organization  members. 
Third,  measures  of  productivity  should  include  characteristics  such  as  validity, 
understandability,  and  reliability  (Howell  and  Van  Sickle,  1982:28-29).  Ghobadian  and 
Ashworth  emphasize  the  following  guidelines:  Performance  measurement  systems  should 
employ  efficiency  and  effectiveness  concepts,  be  able  to  identify  possible  comprimises 
between  different  areas  of  performance,  and  should  not  be  ends  in  themselves  (Ghobadian 
and  Ashworth,  1994:50). 

A  key  factor  in  these  characteristics  and  guidelines  is  that  performance  measures  vary 
from  location  to  location.  This  variation  in  performance  measures  is  due  to  differences  in 
production  processes  required,  age  and  suitability  of  equipment,  and  strengths  and 
weaknesses  inherent  in  the  organization.  These  factors  should  be  taken  into  account  when 
designing  a  performance  measurement  system.  Performance  reports  that  are  standardized 
give  consistency  of  reporting  and  enable  comparison  between  plants.  However ,  the 
information  may  be  misleading  and  irrelevant  because  the  reports  do  not  consider 
variations  between  plants  (Maskell,  1991:26-27). 

After  relating  the  aforementioned  characteristics  and  guidelines  to  the  VOC  standard,  it 
is  apparent  that  the  current  performance  measurement  system  lacks  several  important 
concepts.  The  system  does  not  incorporate  the  concepts  of  productivity,  efficiency,  and 
effectiveness  in  the  VOC  measure.  As  long  as  the  VOC  rate  decreases,  no  questions  are 
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asked  about  how  the  maintenance  tasks  were  performed.  Thus  maintenance  personnel 
could  waste  resources,  use  improper  maintenance  procedures,  or  manipulate  work  orders 
to  achieve  a  low  VOC  rate.  Besides  not  having  concepts  of  productivity,  efficiency,  and 
effectiveness,  the  VOC  standard  is  considered  an  end  in  itself  Actual  VOC  rates  have 
become  the  standards  at  certain  locations,  while  the  significance  of  the  original  target  has 
been  lost.  Also,  the  VOC  standard  does  not  account  for  variations  in  input  at  different 
locations.  Throughout  the  Air  Force,  high  VOC  rates  are  associated  with  poor  vehicle 
maintenance  performance  and  thus  it  is  assumed  that  the  vehicle  maintenance  unit  is  not 
performing  satisfactorily.  In  some  cases  unsatisfactory  performance  is  the  cause,  but  there 
are  many  instances  where  limited  resources,  such  as  money  and  manpower,  drive  the  high 
VOC  rate.  Because  there  are  many  different  operating  environments.  Air  Force  personnel 
should  use  performance  measurement  systems  that  are  able  to  account  for  the  various 
inputs  that  might  constrain  performance  at  different  locations. 

So  far,  we  have  discussed  and  related  the  VOC  standard  to  the  essential  characteristics 
of  an  effective  performance  measurement  system.  The  important  concepts  of  any 
performance  measurement  system  are  productivity,  effectiveness,  and  efficiency.  In  ffie 
following  sections  the  relationships  between  these  concepts  and  the  VOC  standard  are 
discussed. 

Productivity  Concept 

More  than  15  different  definitions  of  productivity  have  been  identified,  leading  to 
confusion  and  misunderstanding  about  its  meaning.  However,  several  definitions  are 


18 


presented  in  this  section  that  meet  the  needs  of  this  research.  Mali  defines  productivity  as 
"the  measure  of  how  well  resources  are  brought  together  in  organizations  and  utilized  for 
accomplishing  a  set  of  results.  Productivity  is  reaching  the  highest  level  of  performance 
with  the  least  expenditure  of  resources”  (Mali,  1978:4-6).  In  other  words,  productivity  is 
>  a  combination  of  efifectiveness  and  efficiency  (Shenhav,  Shrum,  and  Alon,  1994:754).  A 

study  of  the  federal  government  conducted  by  the  Civil  Sendee  Commission,  the  General 
Accounting  Office,  and  the  Office  of  Management  and  Budget  and  Audit  stressed  the 
importance  of  this  definition  and  its  applicability  to  government  organizations  (Mali, 
1978:83). 

Sardana  and  Prem  Vrat  developed  a  more  formal  definition  of  productivity.  They 
contend  that  "productivity  as  an  index  of  system  (subsystem)  performance  indicates  the 
extent  of  actual  accomplishment  of  performance  objective(s)  in  relation  to  the  attainable 
level  in  a  given  external  environment."  They  propose  that  productivity  be  looked  at  in 
broad  perspectives  "as  a  parameter  of  the  performance  of  a  system  to  meet  specific 
objectives  of  measurement"  and,  therefore,  be  interrelated  with  the  meaning  of 
measurement  (Sardana  and  Prem  Vrat,  1987:105).  Productivity  is  the  relationship 
between  output  (goods  or  services  provided)  and  input  (resources  used  in  producing  the 
output)  (Aboganda,  1994:94). 

Productivity  is  also  defined  as  a  "systematic  concept  concerning  the  conversion  of 
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inputs  to  outputs  by  the  system  under  consideration”  (Adam,  Hershauer,  and  Ruch, 

*  1981 : 10).  Figure  3  graphically  portrays  Adam’s  definition  of  productivity:  This  expanded 

definition  is  referred  to  as  "total  productivity."  If  outputs  are  not  related  to  all  inputs 
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Figure  3.  Productivity  Defined  (Adam,  Hershauer,  and  Ruch,  1981:10) 

affecting  output,  an  incomplete  measurement  results.  This  type  of  measure,  called  partial 
factor  productivity,  does  not  account  for  total  firm  productivity  and  should  be  avoided  in 
assessing  the  peformance  of  the  firm  (Lee,  1991 ;  1 1).  For  example,  eflScient  labor  can 
produce  a  large  quantity  of  inventory,  but  if  the  labor  accomplishes  this  task  by  consuming 
more  input  than  needed,  the  end  result  is  counterproductive.  Although  the  labor  is 
efficient,  the  end  result  is  flawed.  VOC  rates  do  not  include  all  the  inputs  required  to 
perform  the  maintenance  function.  These  rates  can  be  reduced  by  spending  additional 
money  for  contracting,  performing  maintenance  that  does  not  meet  safety  requirements,  or 
replacing  parts  instead  of  rebuilding  them. 

Most  definitions  of  productivity  contain  a  ratio  measuring  outputs  against  inputs.  Mali 
not  only  relates  productivity  as  a  ratio,  but  uses  a  productivity  index  to  explain  the 
relationship  between  efficiency  and  effectiveness.  He  states  that  productivity  is  a 
combination  of  effectiveness  and  efficiency.  This  relationship  is  shown  in  Figure  4. 


n  j  ■  A  output  obtained 
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efficiency 

Figure  4.  Productivity  Index  (Mali,  1978:7) 


This  measurement  of  output  to  input  is  essential  in  any  organization. 
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Productivity,  as  defined  for  the  purpose  of  this  paper,  is  both  a  measure  of 
effectiveness  and  efficiency.  This  relationship  is  shown  in  Figure  5.  Current  productivity 
measurement  of  vehicle  maintenance  units  is  expressed  solely  in  terms  of  effectiveness  and 
does  not  account  for  various  input  factors  to  give  a  true  measure  of  productivity. 
However,  there  is  no  concern  about  the  amount  of  resources  expended  in  achieving  the 
performance.  The  VOC  standard  considers  achieving  the  organizational  goal 
(effectiveness),  but  does  not  consider  the  inputs  utilized  (efficiency),  and  thus  no  measure 
of  true  productivity  is  possible. 
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Figure  5.  Components  of  Productivity  (Thom,  1980:31) 


Effectiveness  Concept 

Effectiveness  has  many  definitions  applicable  to  different  operating  environments. 

To  identify  which  definition  or  combinations  of  definitions  is  applicable  to  vehicle 
maintenance  units,  a  review  of  the  most  popular  definitions  is  necessary.  The  legitimacy 
model  measures  effectiveness  based  on  organizational  survival.  Legitimate  activities,  as 
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viewed  by  the  external  public,  are  pursued  by  the  organization  to  ensure  longevity.  This 

model  is  most  useful  on  the  macro  level  when  assessing  which  organizations  are  likely  to 

survive  (Cameron,  1984:278).  Satisfying  the  constituencies  of  an  organization  is  the 

measure  of  effectiveness  for  the  strategic  constituencies  model.  This  model  should  be 

used  to  assess  effectiveness  when  the  constituencies  of  an  organization  have  powerful 

influence  over  the  course  of  action  the  organization  pursues  (Cameron,  1984:277).  The  » 

system  resource  model  emphasizes  the  acquisition  of  needed  resources  to  accomplish 

organizational  goals.  This  model  should  be  used  to  assess  effectiveness  when  there  is  a 

clear  relationship  between  the  resources  received  and  the  tasks  of  the  organization.  Any 

organization  that  acquires  resources  and  stores  them  without  any  intention  of  using  the 

resources  is  increasing  organizational  waste  and  should  not  be  judged  to  be  effective 

(Cameron,  1984:277).  The  last  model  of  effectiveness  is  the  goal  model  This  model 

measures  effectiveness  by  determining  the  extent  an  organization  achieves  its  stated  goals 

and  objectives.  This  model  should  be  used  when  an  organization  has  clearly  defined,  time 

based,  measurable  goals  (Cameron,  1984:276).  Because  VOC  rates  are  time  related, 

clearly  defined,  and  measurable,  the  goal  model  of  effectiveness  will  be  used  for  this 

research.  The  goal  of  the  VOC  rate  is  to  satisfy  the  customer  by  ensuring  there  is  an 

adequate  number  of  vehicles  available  at  all  times. 

From  the  perspective  of  the  customer,  effectiveness  is  “doing  the  right  things  seen 
through  the  customer’s  eyes”  (Theeuwes  and  Adriaansen,  1994:91).  Satisfying  the 
customer  involves  transforming  inputs,  through  a  number  of  value  added  processes,  to 
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outputs.  The  value  added  processes  must  be  considered  important  to  the  customer  for  the 
organization  to  be  effective  (Dumond,  1994A:4).  From  an  organizational  perspective,  this 
concept  is  viewed  as  a  function  of  achieving  company  goals  and  objectives  (Dumond, 
1994:17).  Stated  simply,  effectiveness  is  how  well  a  responsibility  center's  outputs  satisfy 
the  goals  of  the  organization  (Keating  and  Keating,  1981:43). 

The  National  Consumer  Council  of  London,  England  conducted  a  study  which 
emphasized  effectiveness  within  local  government.  Because  local  governments  provide 
services,  the  effectiveness  of  government  agencies  should  be  measured  in  terms  of  the 
“nature,  quantity  and  quality  of  service,  their  effect  and  extent  to  which  they  meet 
consumer  and  community  needs  (Ghobadian  and  Ashworth,  1994:43).”  This  study 
identified  four  questions  that  effectiveness  measures  need  to  address.  First,  effectiveness 
measures  need  to  answer  how  well  the  product  or  service  satisfies  the  customer’s  need. 
Second,  effectiveness  measures  need  to  determine  if  the  product  or  service  is  easy  to  use. 
The  third  question  should  address  the  side  effects  of  the  service  or  product.  Finally,  the 
measure  should  detemune  if  the  product  or  service  is  worthwhile  fi-om  an  economy 
perspective  (Is  the  taxpayer’s  money  being  used  efficiently?)  (Ghobadian  and  Ashworth, 
1994:43). 

Based  on  the  aforementioned  information,  measures  of  vehicle  maintenance 
effectiveness  should  provide  an  indicator  of  the  maintenance  program’s  value,  indicate  the 
extent  to  which  the  right  things  are  being  accomplished,  and  provide  information  to  track 
performance  and  implement  improvements  (Walker  and  Cooper,  1990:28). 
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An  important  theme  present  in  the  definitions  of  effectiveness  and  the  questions  about 
effectiveness  is  the  satisfaction  of  the  customer.  For  the  purpose  of  tMs  research, 
effectiveness  is  defined  as  achieving  both  customer  satisfaction  and  organizational  goals. 
Although  it  was  previously  stated  that  VOC  rates  account  for  effectiveness,  in  reality  there 
is  no  measure  of  effectiveness.  There  is  no  clear-cut  relationship  established  between  a  10 
percent  VOC  rate  and  customer  satisfaction.  Vehicle  maintenance  units  may  attain  the 
organizational  goal  of  10  percent  but  still  not  achieve  true  customer  satisfaction. 

EfSciencv  Concept 

EfBciency  is  a  narrow  and  well-defined  concept  when  compared  to  either  productivity 
or  effectiveness.  Efficiency  refers  to  whether  an  output  is  produced  at  least  cost,  or  a 
given  amount  of  resources  is  used  in  such  a  way  to  produce  the  greatest  results  (Rushing, 
1974:477).  “The  degree  of  conversion  of  a  resource  fi’om  its  initial  form  to  its  final  form 
with  the  help  of  technology,  methods,  and  manpower  available  in  a  given  period”  is 
another  definition  of  efficiency.  This  definition  refers  to  the  level  of  resources  consumed 
in  achieving  results  (Ray  and  Sahu,  1990:154).  Dumond  states  that  the  efficiency  of  an 
organization  can  be  measured  by  the  ratio  of  the  amount  of  resources  used  in  producing  an 
output  (Ehjmond,  1994:17). 

According  to  the  aforementioned  definitions,  efficiency  involves  a  comparison  or  ratio 
of  inputs  utilized  to  outputs  produced  with  no  regard  to  whether  the  output  supports 
organizational  goals.  Therefore,  units  that  utilize  less  resources  when  accomplishing  their 
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tasks  are  more  efficient.  However,  if  the  outputs  do  not  contribute  to  the  attainment  of 
organizational  goals,  these  units  are  not  effective  (Thom,  1980:30;  Mentzer  and  Konrad, 
1991 :34).  The  following  example  illustrates  how  a  unit  can  be  efficient  and  not  contribute 
to  organizational  goals.  Top  management’s  goal  for  an  injection  molding  unit  was  to 
increase  efficiency  and  lower  costs.  The  department  achieved  both  these  goals  by 
producing  more  molds  than  necessary.  The  production  of  extra  molds  led  to  high  labor 
and  machine  utilization  rates  and  low  overhead  costs  because  costs  were  allocated  to  more 
parts.  Although  management  praised  this  unit,  the  unit  actually  added  additional  costs  to 
the  organization  in  the  forms  of  extra  inventory,  increased  energy  bills,  and  increased  wear 
and  tear  of  the  machinery.  Overall,  this  unit  was  efficient,  but  lacked  effectiveness 
(Nemeth,  1991:84). 

For  the  piupose  of  this  research,  efficiency  is  defined  as  the  ratio  of  inputs  consumed  to 
outputs  produced.  As  stated  previously,  VOC  rates  do  not  account  for  efficiency  because 
the  resources  consumed  when  repairing  a  vehicle  are  not  measured. 

Summary  of  Ideas  Presented 

In  this  section  we  developed  five  essential  characteristics  of  a  performance 
measurement  system  and  related  the  current  VOC  standard  to  these  characteristics.  This 
comparison  demonstrated  that  the  VOC  standard  did  not  possess  the  essential 
characteristics  of  an  effective  performance  measurement  system.  Additionally,  the 
relationship  between  VOC  rates  and  the  concepts  of  productivity,  effectiveness,  and 
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efficiency  were  discussed.  The  overall  productivity  of  an  organization  is  measured  by 
outputs  achieved  (effectiveness)  in  meeting  an  objective  with  minimum  use  of  inputs 
(efficiency).  While  efficiency  is  focused  on  how  well  an  organization  uses  resources 
available  to  produce  output,  effectiveness  is  objective-oriented.  Efficiency  and 
effectiveness  are  inter-related  to  productivity;  both  are  focused  on  outputs. 

Current  VOC  rates  strive  to  measure  effectiveness,  but  there  is  no  customer  input  as  to 
what  constitutes  customer  satisfaction.  Thus,  vehicle  maintenance  units  do  not  know  if 
their  operations  are  effective.  There  is  no  measure  of  inputs  consmned  when  producing 
output,  which  leads  to  the  conclusion  that  vehicle  maintenance  units  do  not  know  if  then- 
operations  are  efficient.  If  there  is  no  measure  of  effectiveness  and  efficiency,  true 
productivity  is  impossible  to  determine.  Based  on  the  characteristics  of  a  performance 
measurement  system,  the  concepts  of  productivity,  effectiveness,  and  efficiency,  we 
conclude  that  the  current  VOC  standard  is  not  useful.  In  an  attempt  to  establish  a 
performance  measurement  system  that  would  be  useful  to  vehicle  maintenance  units,  the 
following  section  on  performance  measurement  in  the  government  is  presented. 

Performance  Measurement  In  The  Government 

The  overall  goal  of  this  section  is  to  review  research  about  performance  measurement 
within  the  government  and  suggest  a  possible  performance  measurement  system  for 
vehicle  maintenance  units.  To  accomplish  this  task,  a  review  of  performance  measurement 
within  the  DOD  and  government  agencies  will  be  presented. 
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Although  decision-makers  often  advocate  and  praise  the  use  of  performance 
measurement  within  government,  most  agencies  make  little  use  of  performance 
measurement  systems.  A 1971  study  conducted  by  the  Urban  Institute  concluded  that  of 
thirty  local  governments,  only  23  percent  of  the  performance  measures  included  the 
'  concepts  of  efficiency  and  only  13  percent  included  the  concepts  of  effectiveness 
(Ammons,  1995:41-42).  Follow-on  research  conducted  in  1976  by  the  Urban  Institute 
found  that  of 247  local  governments,  only  10  percent  performance  measures  included  the 
concepts  of  efficiency  and  only  25  percent  included  concepts  of  effectiveness  (Hatry, 
1978:29).  At  the  state  level  (32  states),  47  percent  of  the  respondents  rated  existing 
efficiency  measures  as  barely  adequate  and  91  percent  rated  effectiveness  measures  as 
barely  adequate  (Hatry,  1978:29).  In  1987,  LeGrotte  examined  sbrty  cities  with 
populations  over  100,000  that  had  responsibility  for  police,  fire,  and  library  functions.  The 
results  showed  that  40  percent  of  these  agencies  had  no  performance  measurement 
system.  Of  those  that  had  a  system,  only  45  percent  included  the  concept  of  effectiveness 
and  only  16  percent  included  the  concept  of  efficiency  (Ammons,  1995:42). 

Realizing  the  importance  of  performance  measurement  and  the  lack  of  its  use  within 
government  organizations,  the  DOD  established  two  directives  that  emphasized 
performance  measurement.  In  DOD  Directives  5010.31  and  5010.34,  DOD  Productivity 
Program,  the  government  set  out  to  improve  the  productivity  of  defense  organizations. 
DOD  Directive  5010.3 1  mandates  management  to  focus  on  improving  productivity  by 
using  available  resources  and  improving  methods  of  operations.  DOD  Directive  5010.34 
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mandates  that  defense  organizations  operate  efficiently  and  effectively.  The  DOD  realizes 
there  is  a  need  to  improve  the  efficiency  and  effectiveness  of  its  operations  to  improve 
productivity.  In  a  separate  directive  from  the  Office  of  Management  and  Budget,  dated  9 
July  1973,  the  concept  of  productivity  is  further  emphasized.  The  directive  mandates  the 
establishment  of  a  system  for  measuring  and  evaluating  productivity  in  government 
organizations  (Howell  and  Van  Sickle,  1982:6-7). 

Research  conducted  by  Weisert  and  Clarke  in  1972,  entitled  Determination  of 
Performance  Indicators  For  The  United  States  Air  Force  Base  Level  Transportation 
Function,  found  that  less  than  30  percent  of  the  respondents  were  able  to  reach  agreement 
on  a  desirable  level  of  performance.  Most  of  these  disagreements  were  with  performance 
indicators  presently  in  use  that  were  based  on  generic  Air  Force  standards  such  as  the 
VOC  standard.  In  addition,  the  researchers  concluded  that  the  information  system 
available  to  transportation  managers  (VIMS)  did  not  provide  the  information  needed  or 
desired.  It  was,  therefore,  of  little  use  to  them  in  making  decisions  (Weisert  and  Clarke, 
1972:82-83). 

Research  conducted  in  1979  by  Baumgartel  and  Johnson,  entitled  Productivity 
Meiasiirement  In  A  Base  Level  USAF  Civil  Engineering  Organization,  concludes  that  the 
"productivity  for  a  branch  is  equal  to  the  sum  of  that  branch's  performance  indicators  for 
all  of  its  objectives  and  goals,  divided  by  the  total  number  of  performance  indicators, 
divided  by  the  total  input  to  the  branch”  (Baumgartel  and  Johnson,  1979:29).  They 
conclude  that  this  model  could  effectively  measure  the  productivity  at  the  branch  level 
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because  it  is  based  on  evaluation  of  actual  results,  desired  results  and  inputs.  Because  few 
output  measurement  requirements  are  used  above  base  level,  they  recommend  the  desired 
output  level  (standard)  should  be  established  at  base  level  to  evaluate  performance 
(Baumgartel  and  Johnson,  1979:107-108). 

A  1982  study  by  Howell  and  Van  Sickle,  entitled  Perceptions  of  a  Methodology  for  the 
Development  of  Productivity  Indicators,  reaches  several  conclusions  about  developing 
productivity  indicators  using  criteria  established  by  The  Maryland  Center  for  Productivity 
and  Quality  of  Worklife  study  in  1981.  The  first  conclusion  was  that  the  indicators  were 
more  acceptable  and  meaningful  than  previous  indicators  because  they  incorporated  the 
concepts  of  effectiveness  and  efficiency.  Effectiveness  and  efficiency  measurements 
included  the  characteristics  of  completeness,  comparability,  input  coverage,  compatibility 
with  existing  input  sources,  cost  effectiveness,  consistency  across  organizations,  and 
acceptability  to  organization  members.  The  study  also  determined  that  these  indicators 
might  give  management  a  better  "picture"  of  an  organization’s  productivity  and  pro\ide  a 
better  understanding  of  productivity  and  effectiveness  (Howell  and  Van  Sickle,  1982:77). 

Research  conducted  by  Brewer  in  1989,  entitled  Perceptions  Of  Air  Force  Base-Level 
Transportation  Officers  Towards  The  Effectiveness  Of  Air  Force  Base-Level 
Transportation  Performance  Measurement  Indicators,  reaches  many  of  the  same 
conclusion  as  the  1972  study  by  Weisert  and  Clarke.  The  research  concluded  that  a 
majority  of  transportation  officers  (56.4%)  viewed  many  of  the  current  transportation 
performance  indicators  as  meeting  the  seven  characteristics  of  effective  performance 
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measurement  indicators  established  by  A.  T.  Kearney.  These  characteristics  are  validity, 
coverage,  comparability,  completeness,  usefulness,  compatibility  and  cost  effectiveness. 
The  study  concluded  that  because  there  was  significant  disagreement  (43.6%)  with  the 
performance  indicators,  further  research  should  be  conducted  to  eliminate  or  modify 
indicators  that  do  not  meet  these  characteristics.  In  addition,  a  majority  of  transportation 
officers  perceive  the  necessity  of  including  manpower,  environment,  and  budget  in 
performance  measurement  design  and  they  were  not  in  favor  of  using  current  performance 
indicators  to  cross-compare  organizations.  The  VIMS  system  was  viewed  as  outdated 
because  it  does  not  reflect  "current  performance  measurement  philosophy."  Performance 
standards  need  to  be  designed  and  interpreted  by  accounting  for  unique  operational 
situations.  Included  in  this  sensitivity  are  base  location,  climate,  training  levels  of 
personnel,  and  age  of  the  vehicle  fleet.  Performance  measurement  should  include 
"required  manpower"  as  a  constraint.  Finally,  the  research  concluded  that  if  present 
performance  indicators  are  not  modified  to  include  independent  variables  such  as 
manpower,  base  location,  and  budget,  they  should  not  be  used  to  cross-compare 
organizations  ^rewer,  1989:92-96). 

The  reviewed  research  highlights  the  need  for  a  new  performance  measurement  system 
within  vehicle  maintenance  and  emphasizes  the  following  points: 

1 .  Performance  indicators  within  government  should  include  the  concepts  of 
efficiency  and  effectiveness. 

2.  The  information  available  from  VIMS  does  not  aid  the  decision-making 
process. 

3.  Desired  performance  standards  should  be  determined  at  base  level. 

4.  Performance  measurement  needs  to  account  for  factors  such  as  manpower  and 
budget. 


30 


5.  VOC  rates  should  not  be  used  to  cross-compare  units  at  different  locations 
unless  independent  variables  such  as  manpower,  age  of  fleet,  base  location,  and 
operating  environment  are  accounted  for  in  the  performance  measure. 


Performance  Measurement  Systems  For  Vehicle  Maintenance 

In  an  effort  to  satisfy  the  aforementioned  issues  about  vehicle  maintenance 
performance  measurement,  a  review  was  conducted  to  discover  the  type  of  performance 
measurement  system  that  could  be  useful  for  vehicle  maintenance.  The  performance 
measurement  systems  of  Air  Force  aircraft  maintenance  units  and  the  U.S.  Postal  Service 
were  examined. 

A  study  conducted  in  1991  by  Jung,  entitled  Determining  Production  Capability  In 
Aircraft  Maintenance:  A  Regression  Analysis,  concludes  that,  based  on  the  need  for 
identification  and  consideration  of  independent  variables  affecting  maintenance  constraints 
and  production  outputs,  linear  regression  is  the  appropriate  technique  in  evaluating 
performance  (Jung,  1991:25).  The  findings  of  his  research  are  listed  below: 

1.  The  use  of  productivity  measures  in  evaluating  aircraft  maintenance 
organizations  should  provide  insight  and  be  “tempered”  with  the  fact  that 
aircraft  maintenance  is  a  dynamic  environment. 

2.  Maintenance  managers  using  the  same  performance  measures  for  different 
situations  and  environments  to  assess  performance  may  find  “evaluations 
divergent  from  reality.” 

3.  Maintenance  managers  should  not  evaluate  all  aircraft  using  the  same 
production  measurement  indicators. 

4.  Using  a  set  of  indicators  for  one  aircraft  may  show  high  performance  levels, 
while  using  the  same  measures  for  another  aircraft  may  indicate  low 
performance  levels.  (Jung,  1991:115) 
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Follow-on  research  conducted  by  Gray  and  Ranalli  in  1993,  entitled  An  Evaluation  Of 


Aircraft  Maintenance  Performance  Factors  In  The  Objective  Wing,  concludes  that  the 
method  used  most  often  for  creating  performance  measurement  models  is  regression 
analysis.  They  verified  the  conclusion  made  by  Jung  that  each  aircraft  maintenance  unit 
was  affected  by  different  independent  variables  and  concluded  that  no  single  model  was 
appropriate  for  every  unit.  They  also  concluded  that  regression  models  were  useful  for 
predicting  aircraft  maintenance  performance  measures. 

The  methodology  used  by  Bradley  and  Baron  in  measuring  the  performance  of  the  U.S. 
Postal  Service  involves  multiple  regression  analysis.  They  refer  to  their  model  as 
"operating  efficiency"  and  state  this  method  is  usefiil  for  any  organization  desiring  a  single 
measure  of  performance  and  wanting  to  determine  why  performance  varies  from  place  to 
place  or  over  a  period  of  time  (Appendbc  A)  (Bradley  and  Baron,  1993:450-454). 

The  variables  include  factors  such  as  the  age  of  the  facility,  amount  of  automated 
equipment,  and  facility  size.  Statistical  analysis  determined  which  variables  were  relevant 
factors  influencing  operating  efficiency  and  quantified  the  effect  of  each  variable  on  the 
efficiency  level  of  the  operation.  The  researchers  stressed  the  importance  of  dividing  the 
variables  into  those  that  can  be  controlled  by  management  and  those  that  can  not.  This 
division  of  variables  is  important  when  comparing  performance  at  different  locations.  For 
example,  operating  efficiency  at  an  automated  plant  would  be  higher  than  that  of  an  older, 
low  technology  facility.  Because  all  relevant  variables  are  considered  using  this  approach, 
the  operating  efficiency  of  one  operation  can  be  compared  against  a  "model"  plant  having 
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the  same  characteristics.  This  model  also  allows  for  the  interaction  of  variables  on  one 


another.  An  illustration  given  was  that  operating  efficiency  increases  with  use  of 
automation,  but  decreases  with  the  age  of  the  equipment  (Bradley  and  Baron,  1993:452- 
454).  For  example,  the  U.S.  Postal  Service  used  the  analysis  procedure  to  determine  the 
effect  on  operating  efficiency  if  the  values  of  contributing  factors  were  changed.  Figure  7 
shows  the  change  in  operating  efficiency  due  to  a  10  percent  increase  in  the  factor  listed. 


Factor 

Description 

Degree  of 
Automation 

Percentage  of  piece  handlings  performed  on 
automated  equipment 

9.55 

Volume  of  Mail 

Total  piece  handlings 

2.51 

Age  measured  in  years 

-0.31 

Degree  of  Support 
Costs 

Percentage  of  labor  hours  in  human  resource 
and  training  functions 

1.03 

Space  Utilization 

Number  of  piece  handlings  per  square  foot 
of  mail  processing  space 

0.65 

Degree  of  Flex 

Labor 

Percentage  of  workforce  that  is  classified  as 
part-time  or  casual 

0.37 

Delivery  Network 

Number  of  delivery  points  for  a  given 
volume 

-2.25 

Number  of 

Locations 

Number  of  locations  in  which  mail 
processing  takes  place 

-1.13 

Figure  6.  Factors  Affecting  Operating  Efficiency  (Bradley  and  Baron,  1993:455) 


A  major  benefit  of  this  type  of  performance  measurement  is  that  it  can  be  used  to 
determine  why  operating  efficiencies  vary  fi'om  unit  to  unit  or  over  periods  of  time.  It  can 
also  be  used  to  determine  which  variables  contribute  the  most  to  productivity.  This 
approach  has  aided  managers  in  making  difficult  decisions  such  as  where  to  locate  new 
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facilities,  types  of  capital  investments  to  make,  and  decisions  concerning  performance 
measurements  themselves  (Bradley  and  Baron,  1993:456). 

Ray  and  Sahu  advocate  the  use  of  regression  models  as  a  performance  indicator  for 
productivity,  efiBciency,  and  effectiveness.  They  conclude  that  regression  models  are 
suitable  for  service  organizations  that  want  to  measure  the  effect  of  both  quantifiable  and 
intangible  factors  that  affect  performance  (Ray  and  Sahu,  1990:162). 

Chapter  Summary 

This  literature  review  examined  relevant  information  about  performance  measurement 
systems  and  how  effective  systems  were  related  to  productmty,  effectiveness,  and 
efficiency.  Prior  studies  were  also  examined  to  determine  the  applicability  of  multiple 
regression  models  in  measuring  performance  within  the  government.  These  studies 
revealed  dissatisfaction  with  current  measurement  systems  utilized  in  Air  Force  vehicle 
maintenance  units. 

Specifically,  this  review  identified  that  performance  measurement  systems  within  the 
government  lacked  the  concepts  of  efficiency  and  effectiveness.  Research  focused  on 
vehicle  maintenance  units  also  indicated  this  same  problem.  Additionally,  the  research 
highlighted  the  inadequacies  of  the  VOC  standard  and  stressed  the  importance  of 
developing  a  new  performance  standard  that  allows  for  the  cross-comparison  of  units. 

Research  that  focused  on  USAF  aircraft  maintenance  units  and  the  U.S.  Postal  Service 
indicated  the  usefulness  of  using  multiple  regression  analysis  as  a  form  of  performance 


34 


measurement.  The  benefits  of  multiple  regression  measurement  systems  were  that  they 
aided  in  the  decision-making  process,  allowed  comparison  of  organizations,  could  be  used 
to  determine  causes  of  inefficiencies  within  an  organization,  and  allowed  for  a  single 
measure  of  performance. 
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in.  Methodology 


Introduction 

Given  a  premise  that  current  vehicle  out-of-commission  standards  are  not  indicative  of 
how  a  vehicle  maintenance  unit  is  performing  and  can  not  be  used  to  effectively  cross¬ 
compare  units,  this  chapter  describes  the  methodology  used  to  answer  the  research 
question  presented  in  Chapter  I.  This  question  asked  what  the  best  measure  of  efficiency, 
effectiveness  and  productivity  for  Air  Force  vehicle  maintenance  units  was  that  will  allow 
accurate  cross-comparison  of  these  units.  To  further  define  and  focus  on  this  question, 
eight  investigative  questions  were  developed.  These  are  listed  below. 

1.  How  are  efficiency,  effectiveness  and  productivity  related  to  performance 
measurement? 

2.  Why  are  more  relevant  performance  measurement  indicators  needed  in 
government  organizations? 

3.  How  do  vehicle  maintenance  officers  perceive  the  VOC  standard? 

4.  Should  independent  factors  be  included  when  establishing  a  performance 
standard? 

5.  Is  multiple  regression  analysis  suitable  for  evaluating  the  performance 
of  vehicle  maintenance  units? 

6.  What  independent  variables  impact  the  performance  of  vehicle  maintenance 
units  the  most? 

7.  Are  there  differences  in  perceptions,  based  on  rank,  vehicle  maintenance 
ejqjerience,  and  current  duty  position,  about  an  acceptable  vehicle  maintenance 
performance  standard? 

8.  Are  there  differences  in  perceptions,  based  on  rank,  vehicle  maintenance 
experience,  and  current  position,  about  the  most  important  independent 
variables  impacting  vehicle  maintenance  performance? 

This  chapter  describes  the  population  and  sample,  survey  instruments,  sampling  plan, 
validation  of  the  survey,  data  collection  plan,  survey  analysis,  the  steps  in  formulating  a 


36 


regression  model,  the  statistical  design,  and  the  assumptions  and  limitations  used  to 
answer  investigative  questions  three,  four,  six,  seven,  and  eight.  Further,  a  description  of 
a  regression  model  will  be  presented  along  with  the  methodology  used  to  establish  the 
model  that  should  be  used  by  vehicle  maintenance  units. 

Population  And  Sample 

The  population  for  this  study  is  Air  Force  transportation  officers.  The  sample  tested 
consisted  of  transportation  officers  with  vehicle  maintenance  experience.  These  experts 
answered  investigative  question  number  three.  According  to  AFMPC,  there  were  168 
transportation  officers  with  Special  Experience  Identifier  (SEl),  LK3.  This  SEI  is  used  to 
identify  logistics  officers  with  vehicle  management  experience.  Air  Force  Manual  36-2105 
describes  officers  assigned  this  SEI  as  having  “12  months’  experience  serving  as  a  vehicle 
maintenance,  vehicle  management,  or  vehicle  operations  officer,  including  civil  engineering 
heavy  equipment  (RED  HORSE)  vehicle  m^tenance  shops”  (Department  of  the  Air 
Force,  1994A;296).  Surveys  returned  fi’om  respondents  with  no  vehicle  maintenance 
experience  were  considered  for  inclusion  to  provide  additional  discriminators  for  testing 
purposes. 

Survey  Instruments 

A  self-administered  questionnaire  was  chosen  to  answer  investigative  question  number 
three.  Because  of  the  large  sample  size  and  geographic  diversification  of  officers 
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surveyed,  this  type  of  instrument  was  more  efficient  and  less  costly  to  administer.  Also, 
the  time  required  to  administer  a  mail  survey  was  minimal  when  compared  to  other  survey 
instruments. 

Sampling  Plan 

A  pilot  survey  (Appendix  B)  was  developed  and  administered  to  transportation 
officers  at  the  Air  Force  Institute  of  Technology  to  help  identify  possible  independent 
variables  affecting  vehicle  maintenance  performance.  Cronbach’s  coefficient  alpha  testing 
was  performed  using  SAS  to  determine  the  internal  consistency  of  the  performance  scale. 
An  alpha  coefficient  of  .84  was  obtained,  indicating  a  high  degree  of  internal  consistency 
(Shane,  1995)  (Appendbc  C).  Next,  the  questionnaire  was  sent  to  the  USAF  Survey 
Control  Office,  Military  Personnel  Center,  Randolph  AFB,  Texas.  The  center  approved 
the  survey  with  minor  corrections  based  on  the  Air  University  Sampling  and  Surveying 
Handbook  and  assigned  a  survey  control  number  (Appendix  D).  Once  the  changes  were 
made,  the  final  survey  was  distributed  to  the  population  sample  (Appendix  E).  The 
primary  objective  of  this  survey  was  to  solicit  responses  fi'om  transportation  officers  about 
a  suitable  performance  measurement  standard  for  vehicle  maintenance  units  and  their 
perceptions  about  variables  most  affecting  performance  levels.  The  questionnaire  was 
divided  into  three  major  sections.  Section  I,  questions  one  through  thirteen,  consisted  of 
background  information  and  demographic  data  and  provided  a  smooth  transition  to  the 
more  thought-provoking  questions  in  the  following  sections.  Section  n,  questions  14 
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through  27,  were  designed  to  gather  information  about  a  vehicle  maintenance  performance 
measurement  standard  and  the  independent  variables  affecting  vehicle  maintenance 
performance.  Question  14  asked  respondents  whether  independent  factors  should  be 
included  when  establishing  a  performance  standard,  with  responses  based  on  the  following 
Likert  Scale: 

1  -  Strongly  Disagree 

2  -  Slightly  Disagree 

3  -  Disagree 

4  -  Neutral 

5  -  Slightly  Agree 

6  -  Agree 

7  -  Strongly  Agree 

Questions  15  through  27  included  the  independent  variables  identified  in  the  pilot 
survey  and  asked  the  respondents  to  rate  the  impact  of  these  variables  on  vehicle 
maintenance  performance  using  the  following  Likert  scale: 

1  -Not  At  All 

2  -  Slight  Extent 

3  -  Moderate  Extent 

4  -  Great  Extent 

5  -  Very  Great  Extent 

Section  EH,  question  28,  consisted  of  an  open-ended  question  asking  respondents  to 
describe  independent  variables,  not  already  listed  in  Section  n,  that  afifect  their 
performance  level. 
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Validation 


Validity  of  this  questionnaire  was  accomplished  by  using  an  expert  in  the  area  of 
research  methods  and  an  expert  in  transportation  to  determine  if  the  survey  was  effective 
in  answering  question  number  three.  This  test  yielded  only  minor  problems  and  changes 
that  were  incorporated  into  the  final  survey.  This  procedure  is  an  accepted  means  of 
validating  survey  instruments  (Emory,  1995:149). 

Data  Collection  Plan 

Data  collected  fi-om  the  surveys  consisted  of  opinions.  Opinions  are  considered  to  be 
valid  if  questions  asked  are  clearly  stated  and  understood  by  the  respondents.  This 
validation  was  established  by  the  experts  prior  to  the  administration  of  the  survey  to  the 
population  sample. 

Pre-printed  address  labels  obtained  from  the  Air  Force  world  wide  locator  were 
attached  to  the  full  size  envelopes  to  ensure  timely  distribution  of  the  questionnaires. 
Accompanying  the  questionnaires  were  a  computer  scan  sheet,  cover  letter,  and  a  self- 
addressed,  stamped  return  envelope  which  enabled  prompt  return  of  the  surveys.  The 
questionnaires  were  mailed  from  the  local  United  States  Post  Office  on  19  May,  1995  and 
the  last  survey  was  returned  on  3 1  July,  1995. 

Survey  Analysis 

Section  I  of  the  surveys  was  analyzed  to  determine  the  demographic  data  of  the 
respondents.  Of  special  interest  were  the  rank,  vehicle  maintenance  experience,  and 
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current  position  subgroups  so  that  differences  in  perceptions  could  be  identified.  These 
subgroups  were  selected  because  they  provided  several  different  view  points  on  the 
perceptions  of  how  one  might  perceive  vehicle  maintenance  performance.  For  example, 
an  individual  with  maintenance  experience  knows  what  types  of  factors  impact 
performance  the  most  when  compared  to  someone  with  no  experience.  Additionally,  an 
»  individual  currently  in  a  VMO  position  might  possess  different  perceptions  fi’om  someone 

who  was  a  VMO  in  the  past. 

Question  14  was  analyzed  to  determine  both  the  adequacy  of  the  current  VOC 
standard  and  the  importance  of  including  independent  factors  when  establishing  a 
performance  standard.  Additionally,  questions  15  through  27  were  analyzed  to  determine 
the  relative  importance  of  the  independent  factors  gathered  from  the  pilot  survey.  The 
mean  value  of  each  variable  was  determined  using  STATISTK  and  listed  in  rank  order. 
These  variables  were  be  tested,  fi'om  highest  mean  response  rate  to  lowest  mean  response 
rate,  until  a  statistically  different  mean  response  rate  was  discovered.  This  statistically 
different  response  rate  represented  the  cut-off  point,  and  all  factors  falling  below  were 
excluded  from  further  analysis.  A  second  criteria  for  selecting  factors  that  impact  vehicle 
,  maintenance  performance  the  most  was  that  the  mean  response  rate  must  be  greater  than 

four.  This  criteria  ensured  that  only  the  factors  perceived  as  impacting  vehicle 

* 

maintenance  performance  to  a  “Great  Extent”  or  “Very  Great  Extent”  were  included  for 
further  analysis.  Section  HI  was  analyzed  to  determine  if  other  independent  variables  not 
listed  in  Section  n  were  considered  important  by  the  sample  population.  If  the  same 
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independent  variables  were  listed  by  a  large  proportion  of  respondents  and  were  not 
contained  in  Section  n,  they  were  considered  for  additional  research.  Once  the  most 
relevant  independent  variables  were  established,  collection  of  data  pertaining  to  these 
variables  would  be  possible. 

Formidating  A  Regression  Model 

The  first  step  in  formulating  a  suitable  model  was  to  "hypothesize  the  determimstic 
component  of  the  model  that  relates  the  mean,  E(y),  to  the  independent  variable  x 
(McClave,  1994:461)."  This  was  considered  at  the  time  to  be  vehicle  out-of-commission 
rates.  The  model  would  be  shown  as: 

y  =  Po  +  piXi  +  P2X2+  PsXs  +  p4X4  + -•+  PiXi  (^) 

where:  y  =  vehicle  out-of-commission  rate  =  the  response  variable  we  wish  to 

predict;  P;  are  the  unknown  independent  variable  parameters  and  Xi  are 
information  contributing  variables. 

The  second  step  was  to  estimate  the  unknown  parameters  in  the  model  using  the 
sample  data.  Because  the  number  of  independent  variables  was  expected  to  be  large,  and 
formulation  of  a  regression  model  would  be  difficult  under  these  conditions,  stepwise 
regression  procedures  would  be  used  to  interpret  the  possible  interaction  between 
variables  and  possible  higher  order  terms.  Stepwise  regression  would  result  in  a  "model 
containing  only  those  terms  with  x  values  that  are  significant  at  the  specified  a  level. 
Thus,  in  most  practical  situations,  only  several  of  the  large  number  of  independent 
variables  will  remain”  (McClave,  1994:674-675). 


The  next  step  would  be  to  analyze  the  four  assumptions  of  the  probability  distribution 
associated  with  the  random  error  component,  e.  The  first  assumption  needing  to  be 
satisfied  was  that  the  mean  of  the  probability  distribution  would  equal  zero.  Next,  the 
variance  of  the  probability  distribution  of  8  must  equal  and  be  normal.  Finally,  the 
values  of  8  associated  with  any  two  values  of  y  need  to  be  independent  (McClave, 
1994:472). 

The  fourth  step  would  include  testing  the  usefulness  of  the  model.  Because  stepwise 
regression  analyzes  the  indiwdual  P  parameters  and  includes  only  parameters  that 
contribute  significantly  to  the  model,  evaluation  of  R^,  the  coefficient  of  determination, 
would  determine  the  usefiilness  of  the  model  in  this  step.  "represents  the  proportion  of 

the  total  sample  variability  around  y  that  is  explained  by  the  linear  relationship  between  y 
and  X  • "  An  R^  value  of  1  indicates  all  variability  around  y  is  explained  by  the 
relationship  between  x  and  y  (McClave,  1994:489). 

Statistical  Design 

Although  the  survey  questionnaire  used  an  ordinal  scale  usually  associated  with  the 

calculations  of  percentages,  means,  and  modes,  there  was  evidence  supporting  the  use  of 

an  ordinal  scale  in  the  same  way  as  a  ratio  or  interval  scale: 

In  general,  we  are  perfectly  safe  in  calculating  any  statistic  we  want  on  any  set  of 
measurements  that  have  the  properties  of  an  ordinal  scale.  There  is  definitive 
evidence  that  statistics  calculated  on  an  ordinal  measurements  are  just  as  reliable 
and  meaningful  as  statistics  calculated  on  an  interval  or  ratio  scales  of 
measurements.  (Baker  and  others,  1966:309) 
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Taking  the  conservative  approach,  and  avoiding  the  assumptions  of  an  ordinal  scale,  this 
research  used  nonparametric  tests  to  determine  if  there  were  any  statistically  different 
mean  responses  between  subgroups.  Additionally,  the  non-normality  of  the  responses 
indicated  that  nonparametric  procedures  should  be  used.  A  \^^s-Shapiro  statistic  of  .9 
or  greater  indicates  normality  of  the  data.  However,  not  a  single  response  achieved  .9  or 
higher.  The  nonparametric  test  used  in  this  research  to  compare  subgroups  was  the 
Wilcoxon  Rank  Sum  Test.  This  test  is  almost  as  powerful  as  the  parametric  two-sample 
T-test  and  is  more  powerful  than  the  nonparametric  median  test  (Statistix,  1992: 1 16). 
Because  our  subgroup  populations  were  larger  than  10,  the  use  of  the  Z-statistic  was 
permissible  in  the  \\^coxon  Rank  Sum  Test  (McClave  and  Benson,  1994:930).  The 
following  procedure  was  employed  to  test  each  subgroup.  The  null  hypothesis  (Ho)  stated 
that  the  two  populations  had  identical  response  rates.  The  alternate  hypothesis  (Ha)  stated 
that  the  two  populations  had  different  response  rates.  The  test  statistic  was  calculated  by 
STATISTIX.  The  rejection  region  for  a  two-tailed  test  and  an  a  equal  to  .10  was  jrj  > 

1 .645.  Any  test  statistic  meeting  this  criteria  results  in  a  rejection  of  the  Ho  and  a 
conclusion  that  the  populations  have  different  response  rates  at  a  .10  level  of 
significance.  The  survey  results  indicated  that  the  level  of  significance  was  .0676.  Taking 
the  conservative  approach,  the  research  team  decided  to  use  a  significance  level  of  0. 10. 
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Assumptions  And  Limitations 


This  research  effort  assumed  that  data  collected  from  respondents  was  accurate  and 
reflected  actual  transportation  officers’  perceptions  and  view  points  about  independent 
factors  affecting  vehicle  maintenance  performance  levels. 

Summary 

This  chapter  described  the  research  population  and  sample,  survey  instruments, 
sampling  plan,  validation  of  the  survey,  data  collection  plan,  survey  analysis,  and  the 
statistical  design.  Also  presented  was  a  description  of  a  regression  model,  the 
methodology  used  to  establish  a  useful  model,  and  the  steps  necessary  for  the 
development  of  a  regression  model.  Assumptions  pertaining  to  the  data  collection 
procedure  were  also  stated.  This  methodology  was  used  to  answer  investigative  questions 
three,  four,  sbc,  seven,  and  eight.  Chapter  IV  presents  results  of  our  analysis  based  on  this 
research  methodology  and  provides  information  to  answer  the  aforementioned 
investigative  questions. 
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IV.  Results  And  Analysis 


Introduction 

This  chapter  analyzes  the  data  from  the  survey  questionnaires  and  presents  the  results 
of  the  analysis.  This  chapter  is  divided  into  the  following  sections:  (1)  survey  response 
rate,  (2)  results  of  the  reliability  analysis,  (3)  sample  demographics,  (4)  importance  placed 
on  the  current  VOC  standard,  (5)  percentage  response  results,  (6)  independent  fector 
selection,  (7)  mean  response  results  by  subgroup,  (8)  Wilcoxon  Rank  Sum  Tests  for 
selected  subgroups  and  performance  factors,  (9)  explanation  of  significantly  different 
perceptions,  and  (10)  examination  of  the  comments  from  the  survey  respondents. 
Generalized  test  results  are  presented  in  this  chapter  for  all  statistical  analysis  and  detailed 
results  are  found  in  Appendk  H. 

Survey  Response  Rate 

Surveys  were  sent  to  168  transportation  officers  identified  by  the  Military  Personnel 
Center  (MPC)  as  having  vehicle  maintenance  experience  and  possessing  Special 
Experience  Identifier  (SEI)  LK3.  Although  MPC  identified  168  individuals  as  possessing 
SEI LK3,  15  returned  questionnaires  indicated  that  respondents  had  no  vehicle 
maintenance  experience  (13.4%).  It  was  decided  to  use  these  responses  as  a  means  of 
comparison  for  the  other  data.  112  surveys  were  returned  by  respondents  which  resulted 
in  an  effective  return  rate  of  67  percent.  Additionally,  seven  surveys  were  returned 
because  they  were  undeliverable.  The  confidence  level  for  the  results  of  the  analysis  was 
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calculated  to  be  93.24  percent  based  on  the  aforementioned  sample  size  using  the 
following  formula. 


NZ^  X  .25 

(d^  X  (N-1)  )  +  (Z^  X  .25) 


(2) 


where:  n=  sample  size  (112) 

N  =  total  population  size  (168) 
d  =  precision  or  confidence  level  desired  (5%-  .05) 

Z  =  different  factor  for  each  confidence  level  (unknown) 


Reliability  Analysis  Results 

Although  the  pilot  survey  had  a  Cronbach  alpha  coefficient  of  .84,  indicating  a  high 
degree  of  internal  consistency  and  reliability,  the  research  team  decided  to  test  for 
reliability  using  the  data  from  the  final  survey.  The  internal  consistency  and  reliability  of 
the  final  survey  data  achieved  an  alpha  coefficient  of  .84,  thus  validating  the  results  of  the 
pilot  study. 


Sample  Demographics 

The  entire  listing  of  the  demographic  data  appears  in  Appendk  I.  This  section  reports 
on  specific  demographic  information  useful  as  discriminators  for  this  research. 
Specifically,  5.4  percent  of  the  respondents  were  second  or  first  lieutenants,  33.9  percent 
were  captains,  42.9  percent  were  majors,  and  10.7  percent  were  lieutenant  colonels.  The 
final  7.1  percent  of  respondents  were  in  the  “Other”  category  consisting  of  enlisted  and 
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civilian  vehicle  maintenance  managers.  All  surveys  were  addressed  to  officers  identified 
by  MPC,  but  it  appears  that  maintenance  managers  holdmg  their  positions  at  the  time  the 
surveys  were  distributed  responded  in  their  place.  Of  these  respondents,  92  percent 
possessed  a  transportation  primary  AFSC  and  88.4  percent  possessed  a  transportation 
duty  AFSC.  The  majority  of  respondents  were  from  the  Air  Mobility  Command  (AMC) 
with  31.3  percent,  followed  by  the  Air  Combat  Command  (ACC)  with  13.4  percent. 
Additionally,  a  large  proportion  of  respondents,  33  percent,  came  fi-om  locations  that  were 
not  offered  as  choices  on  the  survey.  Among  others,  these  consisted  of  the  Office  of  the 
Secretary  of  Defense  (OSD),  the  \Clitary  Traffic  Management  Command  (MTMC),  the 
Air  Education  and  Training  Command  (AETC),  the  US  Special  Operations  Command 
(SOC),  and  the  US  Transportation  Command  (TRANSCOM). 

As  part  of  the  methodology,  the  sampling  plan  focused  on  transportation  officers  with 
vehicle  maintenance  experience  to  receive  valid  responses.  For  the  most  part,  the 
sampling  plan  was  successfifi.  Nearly  86.6  percent  of  the  officers  had  at  least  one  or  more 
years  experience  as  a  vehicle  maintenance  officer,  and  only  13.4  percent  had  no  experience 
at  all.  The  analysis  of  this  latter  group  will  be  accomplished  in  a  later  section. 

Importance  Of  The  Current  VOC  Standard 

As  stated  in  Chapter  n,  the  VOC  rate  was  the  primary  performance  indicator  for 
vehicle  maintenance  units.  Data  fi’om  the  survey  respondents  indicated  the  importance  of 
this  performance  measure.  Of  the  transportation  officers  currently  serving  in  positions 
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with  vehicle  maintenance  responsibilities,  88.5  percent  were  required  to  meet  or  exceed 
the  standard  10  percent  VOC  rate.  However,  34.6  percent  of  respondents  were  required 
to  maintain  a  VOC  rate  of  less  than  10  percent.  They  had  adopted  more  stringent 
standards  to  operate  under  and,  although  limited  in  number,  were  found  in  AMC,  ACC, 
and  AFMC.  The  VOC  standard  for  these  respondents  fell  in  a  range  from  four  to  eight 
percent. 

The  VOC  standard  was  not  a  performance  indicator  that  went  unnoticed.  According 
to  the  respondents,  55.5  percent  were  required  to  brief  squadron  commanders^  group 
commanders,  wing  commanders  and  MAJCOM  personnel  on  a  regular  basis.  In  addition, 
77  percent  were  required  to  brief  their  MAJCOM,  67  percent  were  required  to  brief  at  the 
Wing  level,  and  78  percent  were  required  to  brief  VOC  results  at  the  unit  level  on  a 
regular  basis. 

Percentage  Response  Results 

Section  n  of  the  survey  represented  the  primary  area  of  interest  in  this  research  effort. 
The  two  goals  of  this  section  were  to  determine  if  independent  factors  should  be 
considered  when  establishing  an  effective  performance  measurement  standard  and  what 
factors  impact  the  performance  levels  of  vehicle  mmntenance  units  the  most.  Question  14, 
the  first  question  in  this  section,  asked:  “When  manpower  levels  are  established,  various 
factors  are  used  to  determine  what  these  levels  should  be.  To  what  extent  do  you  agree  or 
disagree  that  these  type  of  factors,  such  as  available  manpower,  age  of  vehicle  fleet. 
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budget  levels,  etc.,  should  be  considered  when  establishing  a  vehicle  maintenance 
performance  standard?”  Their  responses  were  based  on  the  following  Likert  scale: 


ptrongly 

Slightly 

Slightly 

Strongly  I 

I  Disagree 

Disagree  Disagree  Neutral 

Agree  Agree 

Agree 

|l - 

...2 - 3 - 4 - 

- 5 - 6 - 

- 7 

Although  vehicle  maintenance  units  are  subject  to  VOC  standards,  a  large  majority  of 
respondents,  88.4  percent,  stated  that  the  standard  should  account  for  independent  factors 
impacting  their  performance  level.  Figure  7  graphically  portrays  the  overwhelming 
number  of  respondents  that  believe  independent  factors  should  be  considered  when 
establishing  a  vehicle  maintenance  performance  standard.  The  complete  breakdown  of 


Figure  7.  Question  14  Response  Breakdown 
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responses  is  shown  in  Table  1.  From  these  responses,  a  performance  standard  that 
includes  such  factors  as  available  manpower,  age  of  vehicle  fleet,  and  budget  levels  should 
be  used  to  measure  vehicle  maintenance  performance. 


Table  2.  Response  Results  for  Question  14. 


Response  Statement 

(7)  Strongly  Agree 

47 

42.0 

42.0 

(6)  Agree 

42 

37.5 

79.5 

(5)  Slightly  Agree 

10 

8.9 

88.4 

(4)  Neutral 

7 

6.3 

94.7 

(3)  Slightly  Disagree 

2 

1.8 

96.5 

(2)  Disagree 

3 

2.7 

99.2 

(1)  Strongly  Disagree 

1 

0.9 

100.0 

Questions  15-27  were  designed  to  determine  how  respondents  viewed  the  impact  of 
various  independent  &ctors  on  the  performance  of  vehicle  maintenance  units.  All 
respondents  were  asked  to  reply  to  the  following  statement;  “To  what  extent  do  you 
believe  each  of  the  following  independent  factors  impact  the  performance  level  of  a  vehicle 
maintenance  unit?”  All  questions  were  answered  using  the  following  Likert  scale: 


Not  Slight  Moderate  Great  Very  Great 

At  All  Extent  Extent  Extent  Extent 

1 - -2 - 3 - 4 - 5 


Responses  to  these  questions  are  shown  in  Table  3.  The  factors  receiving  the  most  ratings 
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Table  3.  Vehicle  Maintenance  Performance  Measurement  Factors 


Independent  Factors 

1^ 

1^ 

1^ 

m 

15.  Available  manpower 

0.0 

0.0 

4.5 

48.2 

47.3 

16.  Training  level  of  assigned  personnel 

0.0 

0.0 

4.5 

42.0 

53.6 

17.  Experience  (#  of  years)  of  assigned  persormel 

0.9 

1.8 

18.8 

46.4 

32.1 

18.  Budget  available  for  vehicle  maintenance 

0.0 

0.9 

12.5 

35.7 

50.9 

19.  Age  of  vehicle  fleet 

0.9 

0.9 

25.0 

33.0 

40.2 

20.  Parts  availability 

0.0 

0.0 

11.6 

33.9 

54.5 

21.  Severity  of  climate 

1.8 

12.5 

48.2 

30.4 

7.1 

22.  Availability  of  warranty  service 

6.3 

25.0 

45.5 

19.6 

3.6 

23.  Age  of  maintenance  facility 

4.5 

24.1 

45.5 

22.3 

3.6 

24.  Size  of  maintenance  facility 

1.8 

14.3 

40.2 

33.9 

9.8 

25.  Tool  &  equipment  availability 

0.9 

0.0 

13.4 

45.5 

40.2 

26.  Forward  operating  location  responsibilities 

u 

12.5 

40.2 

33.0 

11.6 

27.  Vehicle  utilization  rates 

0.9 

8.0 

28.6 

42.0 

20.5 

of  “Very  Great  Extent”  were  parts  availability  (54.5%),  training  level  of  assigned 
personnel  (53.6%),  and  the  budget  available  for  vehicle  maintenance  (50.9%). 
Accordingly,  these  categories  also  received  the  fewest  number  of  ratings  (0%)  in  the  “Not 
At  All”  category.  On  the  other  hand,  the  factors  receiving  the  most  ratings  of  “Not  At 
All”  were  availability  of  warranty  service  (6.3%),  age  of  maintenance  facility  (4.5%),  and 
forward  operating  location  responsibilities  (2.7%).  Mean  response  rates  for  all  the  factors 
are  shown  in  Table  4. 
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Table  4.  Rank  Order  of  Independent  Factors 


Performance  Measurement  Factors 

Mean  Response  Rate 

Training  Level  of  Assigned  Personnel 

4.49 

4.43 

Available  Manpower 

4.43 

Budget  Available  for  Vehicle  Maintenance 

4.37 

Tool  and  Equipment  Availability 

4.24 

Age  of  Vehicle  Fleet 

4.11 

Experience  (#  of  Years)  of  Assigned  Personnel 

4.07 

Vehicle  Utilization  Rates 

3.73 

Forward  Operating  Location  responsibilities 

3.38 

Size  of  Maintenance  Facility 

3.36 

Severity  of  Climate 

3.29 

Age  of  Maintenance  Facility 

2.96 

Availability  of  Warranty  Service 

2.89 

Table  4  shows  that  respondents  believe  training  levels  of  assigned  personnel  (4.49  ) 
affected  performance  levels  of  vehicle  maintenance  units  the  most.  Parts  availability  (4.43) 
and  available  manpower  (4.43)  were  the  next  two  factors  impacting  performance  to  the 
greatest  extent.  Parts  availability  was  listed  first,  among  the  two,  because  this  factor 
received  more  ratings  in  the  “Very  Great  Extent”  category  (54.5%)  when  compared  to 
available  manpower  (47.3%).  The  lowest  rated  factors  were  severity  of  climate  (3.29), 
age  of  maintenance  facility  (2.96),  and  availability  of  warranty  service  (2.89). 


Independent  Factor  Selection 

The  selection  of  independent  factors  for  analysis  and  inclusion  for  determination  of  an 
acceptable  vehicle  maintenance  standard  was  decided  upon  through  the  testing  described 
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in  Chapter  IH.  This  included  Wilcoxon  Rank  Sum  Tests  to  determine  significant 
differences  between  independent  factor  mean  ratings.  The  methodology  also  included 
selection  of  all  mean  responses  falling  between  ratings  of  “Great  Extent”  and  “Very  Great 
Extent.”  The  findings  are  shown  in  Table  5.  This  table  indicates  there  was  not  a 
significant  difference  between  mean  response  rates  until  the  “Experience  (#  of  years)  of 
Assigned  Personnel”  and  “Vehicle  Utilization  Rates”  factors  were  compared.  Because  of 
these  results,  only  the  first  seven  independent  factors  were  considered  for  further  analysis 


Table  5.  Independent  Factor  Analysis 


Independent  Factor 


Independent  Factor 


P-Value 


Z-Value 


Training _ 

Parts  Availability _ 

Available  Manpower _ 

Available  Budget _ 

Tool  &  Equipment  Availability 

Age  of  Vehicle  Fleet _ 

Experience  Level  of  Persoimel 


Parts  Availability _ 

Available  Manpower _ 

Available  Budget _ 

Tool  &  Equipment  Availability 

Age  of  Vehicle  Fleet _ 

Experience  Level  of  Personnel 
Vehicle  Utilization  Rates 


0.7508 

0.6929 

0.8641 

0.2092 

0.3335 

0.6983 

0.0070 


0.318 

0.395 

0.171 

1.256' 

0.967 

0.388 

2.699 


Reject 

HO: 

No 

No 

No 

No 

No 

No 

Yes 


The  results  of  this  testing  met  the  second  criteria  for  inclusion  of  independent  factors  for 
consideration—that  of  achieving  a  mean  rating  of  at  least  4.0. 


Mean  Response  Results  Bv  Subgroups 

Demographic  data  collected  fi'om  this  survey  allowed  for  the  comparison  of  different 
subgroups  and  their  perceptions  about  whether  or  not  independent  factors  should  be 
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considered  when  establishing  a  performance  standard.  This  data  was  also  used  to 
discriminate  among  the  independent  factor  ratings.  The  subgroups  providing  insight  into 
various  perceptions,  as  stated  in  Chapter  El,  were  distinguishable  by  rank,  VMO 
experience,  and  whether  or  not  the  respondent  had  vehicle  maintenance  responsibilities  at 
the  time  the  survey  was  administered.  Breakdowns  of  these  subgroups  are  in  Tables  6  to 
1 1 .  The  mean  values  of  the  “0-1  thru  0-2”  and  “Other”  subgroups  about  the  importance 
of  considering  independent  factors  when  establishing  a  performance  measurement  standard 
were  6.5  and  6.75  respectively  and  fell  in  a  range  of  responses  between  “Agree”  and 


Table  6.  Rank  Subgroup  Mean  Response  vs.  Question  14 


Rank 

Responses 

Mean  Value 

0-1  thru  0-2 

6 

6.5 

0-3 

38 

5.84 

0-4 

48 

5.96 

0-5 

12 

5.92 

Other 

8 

.  .6,75  .  J 

“Strongly  Agree”  on  the  scale.  However,  the  0-3, 0-4,  and  0-5  subgroup  mean  responses 
were  5.84,  5.96,  and  5.92  respectively  and  fell  in  the  range  of  responses  between  “Slightly 
Agree”  and  “Agree.”  To  help  explain  this  finding,  testing  was  performed  to  determine 
why  there  were  differences  in  perceptions;  it  was  discovered  that  every  respondent  in  the 
“0-1  thru  0-2”  and  “Other”  subgroups  was  currently  in  a  VMO  position.  To  determine  the 
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extent  of  the  subgroups’  differences,  Wilcoxon  Rank  Sum  Tests  were  accomplished  in  a 
later  section. 

Analysis  of  the  responses  from  the  second  subgroup,  VMO  experience,  was  performed 
to  determine  if  any  differences  existed  among  respondents.  The  results  of  this  analysis  are 
shown  in  Table  7.  There  were  no  noticeable  differences  in  any  categories  except  those 
from  respondents  possessing  1-2  years  of  VMO  experience.  However,  further  research 


Table  7.  VMO  Experience  vs.  Question  14 


Experience  Level 

Responses 

Mean  Value 

None 

15 

6.1333 

7 

6.0000 

1  but  less  than  2  years 

35 

5.7714 

2  years  or  more 

55 

6.1132 

found  that  of  the  35  respondents  in  this  categoiy,  only  eight  currently  had  VMO 
responsibilities  at  the  time  the  survey  was  administered.  Except  for  respondents  with  1-2 
years  of  experience,  all  other  categories  had  mean  responses  between  “Agree”  and 
“Strongly  Agree”  (6.1333, 6.0000, 6. 1132).  The  respondents  with  1-2  years  of  VMO 
experience  had  a  mean  response  of  5.7714.  This  response  corresponded  to  a  rating 
between  “Slightly  Agree”  and  “Agree.”  Analyzing  the  extent  of  the  subgroups’ 
differences  required  the  use  of  the  Wilcoxon  Rank  Sum  Test  to  determine  if  there  was  a 
statistically  significant  difference  in  perception.  Results  of  this  test  are  reported  in  a  later 
section  of  this  chapter. 
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Analysis  of  the  third  subgroup,  whether  or  not  respondents  had  VMO  responsibilities 
at  the  time  the  survey  was  administered,  provided  valuable  insight  into  how  each  group 
viewed  the  importance  of  using  independent  factors  when  establishing  an  acceptable 
performance  measurement  standard.  Table  8  provides  a  breakdown  of  how  each  category 
responded.  Respondents  with  VMO  responsibilities  had  a  mean  response  of  6.48 


Table  8.  VMO  Responsibilities  vs.  Question  14 


Vehicle  Maintenance 
Responsibilities 

Number  of  Responses 

Mean  Value 

Yes 

25 

6.48 

No 

87 

5.86 

TOTAL 

112 

6.00 

(between  “Agree”  and  “Strongly  Agree”),  while  respondents  that  did  not  have  VMO 
responsibilities  had  a  mean  response  of  5.86  (between  “Slightly  Agree”  and  “Agree”). 
Analyzing  the  extent  of  the  subgroups’  differences  required  the  use  of  the  Wilcoxon  Rank 
Sum  Test  to  determine  if  there  was  a  statistically  significant  difference  in  perception. 
Results  of  this  test  are  reported  in  a  later  section  of  this  chapter. 

It  is  apparent  fi’om  the  analysis  of  the  subgroups  that  a  major  determinant  affecting 
perceptions  about  the  establishment  of  performance  measurements  was  whether  or  not  the 
respondent  had  vehicle  maintenance  responsibilities  at  the  time  the  survey  was 
administered.  This  conclusion  made  sense,  because  a  VMO  would  be  more  intimately 
involved  with  problems  encoimtered  when  dealing  with  the  VOC  performance  standard. 
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Therefore,  VMOs  would  more  likely  possess  a  higher  mean  response  when  asked  to 
determine  if  independent  factors  should  be  included  in  a  performance  measurement 
system. 

In  addition  to  analyzing  the  relationship  between  independent  fectors  and  the 
establishment  of  a  performance  measurement  system,  demographic  data  collected  from 
this  survey  allowed  further  analysis  of  the  independent  factors  presented  in  questions  15- 
27.  The  same  subgroups  were  used  to  analyze  the  seven  independent  factors  chosen  for 
further  study.  Breakdowns  ofthese  subgroups  are  in  Tables  9-11.  As  seen  by  the  mean 
value  responses  in  table  9,  the  subgroups  perceived  each  performance  measurement  factor 
as  impacting  vehicle  maintenance  somewhere  between  a  “Great  Extent”  and  “Veiy  Great 
Extent”  with  five  exceptions.  The  0-1  through  0-2  category  perceived  the  impact  of  tool 
and  equipment  availability  and  experience  (#  of  years)  of  assigned  personnel  quite 
differently  from  the  majority.  The  mean  responses  for  these  factors  were  3.83  and  3.67 
respectively  and  fell  between  the  “Moderate  Extent”  and  “Great  Extent”  range  on  the 
Likert  scale.  The  0-3  subgroup  perceived  the  impact  of  age  of  the  vehicle  fleet  and 


Table  9.  Mean  Response  of  Rank  Subgroup  vs.  Performance  Factors 


Performance  Measurement  Factors 

0-1/0-2 

0-3 

0-4 

0-5 

Other 

Training  Level  of  Assigned  Personnel 

4.50 

4.50 

4.46 

4.50 

4.63 

Parts  Availability 

4.17 

4.47 

4.33 

4.67 

4.63 

Available  Manpower 

4.17 

4.39 

4.48 

4.25 

■  4.75 

Budget  Available  for  Vehicle  Maintenance 

4.83 

4.34 

4.31 

4.17 

4.75 

Tool  and  Equipment  Availability 

3.83 

4.34 

4.17 

4.17 

4.63 

Age  of  Vehicle  Fleet 

4.50 

3.87 

4.29 

4.17 

3.75 

Experience  (#  of  Years)  of  Assigned  Personnel 

3.67 

3.95 

4.10 

4.25 

4.50 
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experience  (#  of  years)  of  assigned  personnel  differently  from  the  majority.  The  mean 
responses  for  these  fectors  are  3.87  and  3.95  respectively  and  fall  between  the  Moderate 
and  Great  Extent  range  on  the  Likert  scale.  The  other  subgroup’s  mean  response  for  age 
of  vehicle  fleet  is  3.75.  Both  of  these  ratings  indicate  a  difference  in  perceptions  and  fell 
between  the  “Moderate”  and  “Great  Extent”  range  on  the  Likert  scale. 

Table  10  displays  the  mean  responses  of  the  VMO  experience  subgroups  in  relation  to 
each  independent  factor.  The  mean  value  responses  indicate  that  the  majority  of 
subgroups  perceive  each  performance  measurement  factor  as  impacting  vehicle 


Table  10.  Mean  Response  of  Experience  Subgroup  vs.  Performance  Factors 


Performance  Measurement  Factors 

None 

eri 

Training  Level  of  Assigned  Personnel 

4.73 

4.44 

4.43 

"4.47' 

Parts  Availability 

4.40 

4.33 

4.29 

4.55 

Available  Manpower 

4.67 

4.11 

4.31 

4.49 

Budget  Available  for  Vehicle  Maintenance 

4.40 

4.33 

4.37 

4.36 

Tool  and  Equipment  Availability 

4.53 

4.00 

4.06 

4.32 

Age  of  Vehicle  Fleet 

3.93 

4.67 

4.03 

4.11 

Experience  (#  of  Years)  of  Assigned  Personnel 

4.27 

3.11 

4.14 

4.13 

maintenance  somewhere  between  a  “Great  Extent”  and  “Very  Great  Extent.”  However, 
there  is  some  disagreement  among  the  subgroups  as  to  how  they  perceive  the  impact  of 
age  of  vehicle  fleet  and  experience  (#  of  years)  of  assigned  personnel.  Personnel  with 
greater  than  zero  but  less  than  one  year  experience  perceived  experience  (#  of  years)  of 
assigned  personnel  as  having  less  impact  on  vehicle  maintenance  performance  than  the 
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other  subgroups.  The  mean  response  for  this  factor  was  3.11  and  corresponded  to  a 
rating  between  “Moderate  Extent”  and  “Great  Extent”  on  the  Likert  scale.  The  zero  year 
subgroup  perceived  age  of  the  vehicle  fleet  differently  than  the  majority.  The  mean 
response  was  3.93  and  fell  between  “Moderate  Extent”  and  “Great  Extent”  on  the  Likert 
scale.  Further  analysis  of  these  subgroups  to  determine  if  the  differences  in  perceptions 
are  statistically  significant  will  be  accomplished  in  the  next  section. 

The  final  discriminating  characteristic  is  personnel  with  VMO  responsibilities  compared 
to  personnel  without  VMO  responsibilities  at  the  time  the  survey  was  administered.  Table 
1 1  displays  the  results  of  this  comparison.  Both  subgroups  agreed  on  every  performance 
factor.  Mean  responses  were  between  4.01  and  4.52  and  represent  a  rating  between 
ranges  of  “Great  Extent”  and  “Very  Great  Extent”  on  the  Likert  scale 


Table  1 1.  Mean  Response  of  VMO  Responsibilities  vs.  Performance  Factors 


Performance  Measurement  Factors 

No  VMO 

Yes  VMO 

Training  Level  of  Assigned  Personnel 

4.48 

4.52 

Parts  Availability 

4.41 

4.48 

Available  Manpower 

4.43 

4.44 

Budget  Available  for  Vehicle  Maintenance 

4.34 

4.44 

Tool  and  Equipment  Availability 

4.23 

4.28 

Age  of  Vehicle  Fleet 

4.10 

4.12 

Experience  (#  of  Years)  of  Assigned  Personnel 

4.01 

4.28 
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Wilcoxon  Rank  Sum  Tests  for  Selected  Subgroups  And  Performance  Factors 


This  section  determines  if  the  differences  in  perceptions  of  the  various  subgroups 
identified  in  the  previous  section  are  statistically  significant.  If  the  differences  in 
perceptions  of  the  subgroups  were  determined  to  be  statistically  significant,  an  explanation 
of  the  differences  is  offered  in  the  next  section  by  the  research  team. 

When  asking  whether  or  not  independent  factors  should  be  considered  when 
establishing  a  performance  standard,  the  rank  subgroups  of  0- 1/0-2  and  others  differed  in 
mean  response  fi’om  the  0-3, 0-4,  and  0-5  subgroups.  Accordingly,  a  "Wlcoxon  Rank  Sum 
Test  was  performed  and  a  z  statistic  and  corresponding  p-value  were  calculated  as  2.04 
and  .041  respectively.  These  values  led  to  the  rejection  of  the  null  hypothesis  and  a 
conclusion  that  the  two  subgroups  have  different  mean  responses. 

The  VMO  experience  subgroups  of  1-2  years  of  experience  differed  in  mean  response 
fi-om  the  none,  less  than  one,  and  two  or  more  years  of  experience  subgroups.  The  results 
of  the  Wilcoxon  Rank  Sum  Test,  1.186  for  a  z  statistic  and  .2355  for  a  p  value,  led  to  the 
non-rejection  of  the  null  hypothesis  and  the  conclusion  that  the  two  subgroups  have  the 
same  mean  responses. 

Analysis  of  data  fi’om  respondents  currently  serving  in  a  position  with  vehicle 
maintenance  responsibilities  shows  a  significant  difference  between  those  respondents  that 
do  not  currently  have  those  same  responsibilities.  The  Wilcoxon  Rank  Sum  Test  resulted 
in  a  z  statistic  of  2. 14  and  a  p-value  of  .032.  This  result  led  to  the  rejection  of  the  null 
hypothesis,  and  a  conclusion  that  each  sample  population  has  different  perceptions  about 
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whether  independent  factors  should  be  considered  when  establishing  an  acceptable 
performance  measurement  standard. 

There  were  disagreements  between  subgroups  when  asking  what  independent  factors 
impact  vehicle  maintenance  the  most.  Table  12  displays  the  independent  factors  and  the 
subgroups  that  needed  to  be  tested. 


Table  12.  Subgroups  For  Testing  Corresponding  Factors 


Performance  Measurement  Factors 

1.  Tool  and  Equipment  Availability 

0-1  thru  0-2 

All  others 

2.  Age  of  Vehicle  Fleet 

0-1/0-2/0-4/0-5 

O-3/Other 

3.  Age  of  Vehicle  Fleet 

4.  Experience  (#  of  Years)  of  Assigned  Personnel 

Less  than  1  Year 

All  others 

5.  Experience  (#  of  Years)  of  Assigned  Personnel 

0-1/0-2/0-3 

0-4-other 

Five  tests  were  needed,  and  the  factors  disagreed  upon  the  most  were  experience  (#  of 
years)  of  assigned  personnel  and  age  of  the  vehicle  fleet.  Wilcoxon  Rank  Sum  Tests  were 
performed  to  determine  if  these  responses  were  significantly  different  in  mean  responses 
between  subgroups.  The  results  of  the  tests  led  to  the  rejection  of  the  null  hypothesis 
(subgroup  mean  responses  were  equal)  on  two  occasions.  The  test  results  are  shown  in 
Table  13. 

The  two  tests  resulting  in  significantly  different  perceptions  of  subgroups  corresponded 
to  the  age  of  vehicle  fleet  and  experience  of  assigned  personnel.  For  the  age  of  the  vehicle 
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fleet  factor,  the  subgroup  of  0-1/0-2/0-4/0-5  differed  fi’om  the  respondents  that  were  0- 
3/Others.  The  experience  (#  of  years)  of  assigned  personnel  factor  differed  significantly 


Table  13.  Results  of  Wilcoxon  Rank  Sum  Tests 


Performance  Measurement  Factors 

z  statistic 

p-value 

reject  Ho 

1 .  Tool  and  Equipment  Availability 

1.157 

.2475 

No 

2.  Age  of  Vehicle  Fleet 

2.280 

.0226 

Yes 

3.  Age  of  Vehicle  Fleet 

.0540 

.9569 

No 

4.  Experience  (#  of  years)  of  Assigned  Personnel 

2.788 

.0053 

Yes 

5.  Experience  (#  of  years)  of  Assigned  Personnel 

1.159 

.2466 

No 

between  the  less  than  one  year  subgroup  (VMO  experience)  and  the  zero,  one,  and  two  or 
more  year  subgroup. 

Explanation  Of  Significantly  Different  Perceptions 

There  were  four  significantly  different  mean  responses  between  subgroups.  The  first 
significant  difference  in  perceptions  occurred  between  the  0-1/0-2/others  subgroup  and  0- 
3/0-4/0-5  subgroup.  This  difference  corresponded  to  whether  or  not  independent  factors 
should  be  considered  when  establishing  a  performance  standard.  The  first  subgroup 
placed  more  importance  on  including  independent  factors  when  compared  to  the  second 
subgroup.  As  stated  earlier,  further  analysis  of  the  first  subgroup  revealed  that  every 
single  respondent  in  the  first  subgroup  was  currently  in  a  position  that  had  VMO 
responsibilities  at  the  time  the  survey  was  administered.  This  realization  implied  that 
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respondents  with  VMO  responsibilities  recognized  the  importance  of  independent  factors 
that  impact  vehicle  maintenance  performance  and,  at  the  time,  were  more  intimately 
involved  with  the  VOC  standard  than  the  rest  of  the  respondents.  Respondents  with 
VMO  responsibilities  were  more  aware  of  the  importance  of  independent  factors  because 
they  were  subject  to  the  daily  demands,  conditions,  and  requirements  associated  with  the 
factors  that  impact  vehicle  maintenance.  For  example,  on  a  daily  basis  a  VMO  had  to  be 
aware  of  budget  levels,  manpower  requirements,  and  training  demands  placed  on  his  or 
her  operation.  Respondents  without  VMO  responsibilities  were,  in  most  cases,  much 
farther  removed  from  the  unique  daily  demands  placed  on  VMOs.  Another  reason  for 
this  difference  was  that,  for  the  most  part,  respondents  with  VMO  responsibilities  were 
located  at  the  squadron  level  where  performance  measurement  was  an  important  concept 
in  the  TQM  movement  in  the  Air  Force.  Personnel  subject  to  TQM  had  received  training 
on  the  importance  of  establishing  performance  measures. 

The  second  significant  difference  in  perceptions  occurred  between  the  respondents  with 
VMO  responsibilities  subgroup  and  the  respondents  without  VMO  responsibilities 
subgroup.  This  difference  corresponded  to  whether  or  not  independent  factors  should  be 
considered  when  establishing  a  performance  standard.  The  first  subgroup  placed  more 
importance  on  including  independent  factors  when  compared  to  the  second  subgroup.  As 
stated  in  the  previous  paragraph,  this  difference  in  perception  should  have  been  expected 
because  of  the  above-mentioned  reasons. 
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The  third  significant  difference  in  perceptions  occurred  among  the  respondents  with  less 
than  one  year  ©cperience  as  a  VMO  and  the  subgroup  of  respondents  with  none,  one,  or 
two  or  more  years  of  experience  as  a  VMO.  The  latter  group  had  a  higher  mean  response 
for  the  impact  of  experience  (#  of  years)  of  assigned  personnel  on  vehicle  maintenance 
performance.  In-depth  analysis  of  this  difference  in  responses  resulted  in  the  discovery  of 
a  significant  difference  between  mean  values  based  on  rank.  The  less  than  one  year 
subgroup  consisted  of  nine  respondents.  Five  of  these  respondents  were  majors  and  the 
remaining  four  were  split  evenly  between  captains  and  lieutenants.  The  mean  response 
rates  for  majors  were  3.8  and  the  captains  and  lieutenants  were  2.25  indicating  a  difference 
in  perception  based  on  time  in  the  service.  Majors,  for  the  most  part,  have  between  1 1 
and  15  years  of  service,  whereas  captains  and  lieutenants  have  much  less  service  time. 
Because  of  this  increased  time  in  the  service,  majors  have  significantly  more  knowledge 
and  experience  in  transportation  related  matters  than  lieutenants  and  captains. 

The  last  significant  difference  in  perceptions  occurred  among  the  0-1/0-2/0-4/0-5 
subgroup  and  O-3/Other  subgroup.  This  difference  corresponded  to  the  impact  of  the  age 
of  vehicle  fleet  factor.  The  former  group  had  a  higher  mean  response  for  the  impact  of 
age  of  vehicle  fleet  on  vehicle  maintenance  performance.  Further  analysis  of  this 
difference  in  mean  response  ratings  revealed  that  16  respondents  in  the  0-1/0-2/0-4/0-5 
subgroup  (24.25%)  were  currently  serving  in  positions  with  vehicle  maintenance 
responsibilities  while  only  nine  respondents  (19.56%)  in  the  O-3/Other  subgroup  were  in 
similar  positions.  Furthermore,  the  0-1/0-2/0-4/0-5  subgroup  averaged  between  one  and 
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two  years  of  VMO  experience  and  the  O-3/Other  subgroups  averaged  one  year  or  less  of 
VMO  ejq)erience.  These  results  lead  to  the  conclusion  that  increased  experience, 
knowledge,  and  time  in  the  Air  Force  lea  to  increased  emphasis  on  independent  factors 
affecting  performance  levels  of  vehicle  maintenance  units. 

Examination  Of  Comments  From  Survey  Respondents 

The  main  purpose  of  section  Id,  question  28,  was  to  obt^  additional  information 
about  factors  impacting  the  performance  of  vehicle  maintenance  units.  This  section  was 
included  because  the  researchers  realized  that  there  might  be  additional  independent 
fectors  that  were  not  included  in  the  survey.  This  open-ended  question  also  provided  a 
forum  for  transportation  officers  to  voice  any  additional  comments  they  might  have. 

Of  the  1 12  surveys  returned,  44  of  them  contained  responses  to  question  28. 

Basically,  there  were  two  types  of  responses.  The  first  type  provided  additional  factors 
that  impact  vehicle  maintenance  performance  not  contained  in  questions  15-27.  The 
factors  appearing  the  most  were:  types  of  Wing  mission,  types  of  customers,  deployment 
taskings,  fleet  composition,  accident  and  abuse  rates,  additional  duties,  condition  of  the 
maintenance  facility.  Structure  and  layout  of  the  maintenance  facility,  time  to  order  and 
receive  parts,  availability  of  commercial  services  and  vendors,  and  strength  of  the  Vehicle 
Control  Officer  (VCO)  program.  After  reviewing  these  responses,  the  research  team 
decided  that  the  responses  with  no  overlap  with  any  of  the  factors  already  in  the  survey 
would  be  considered  for  future  analysis.  Based  on  this  criteria,  strength  of  the  VCO 
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program,  fleet  composition,  and  availability  of  commercial  services  and  vendors  should  be 
studied  in  future  research  efibrts.  These  three  responses  were  prevalent  throughout  the 
survey  responses  for  question  28. 

The  second  type  of  response  described  various  types  of  performance  standards  that 
respondents  perceived  as  adequate  or  wished  to  implement  in  the  future.  The  only 
standard  mentioned  as  adequate  was  the  percent  of  vehicles  returned  fi’om  maintenance 
within  eight  hours.  The  researchers  were  unable  to  locate  this  standard  in  any  vehicle 
maintenance  regulation,  leading  to  the  conclusion  that  this  standard  is  a  newly  adopted 
one  implemented  under  the  TQ  philosophy.  Without  additional  information  about  this 
standard,  no  analysis  or  conclusions  can  be  made  about  the  adequacy  of  the  standard. 

Respondents  also  suggested  performance  standards  they  would  like  to  see 
implemented.  One  respondent  wanted  to  create  a  performance  standard  that  ties  vehicle 
procurement,  vehicle  services,  and  mission  together.  Another  respondent  wanted  to 
create  a  performance  standard  relating  to  vehicle  mission  essential  levels. 

Summary 

This  chapter  presented  the  results  and  analysis  of  the  data  used  to  answer  investigative 
questions  three,  four,  sk,  seven,  and  eight.  The  large  amount  of  data  collected  fi'om  the 
survey  indicated  that  the  current  VOC  performance  standard  was  not  considered  an 
adequate  performance  measure.  The  respondents  overwhelmingly  agreed  that 
independent  factors  need  to  be  included  when  determining  vehicle  maintenance 
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performance.  The  independent  factors  chosen  most  important  by  the  respondents  were 
training  levels  of  assigned  personnel,  parts  availability,  available  manpower,  budget 
available  for  vehicle  maintenance,  tool  and  equipment  availability,  age  of  vehicle  fleet,  and 
experience  (#  of  years)  of  assigned  personnel.  The  only  diJBference  in  perceptions,  based 
on  rank,  vehicle  maintenance  experience,  and  current  responsibilities,  occurred  at  question 
14  (whether  or  not  independent  factors  should  be  considered  when  establishing  a 
performance  standard),  question  17  (experience  (#  of  years)  of  assigned  personnel),  and 
question  19  (age  of  the  vehicle  fleet).  Conclusions  and  recommendations  based  on  this 
analysis  are  presented  in  Chapter  V. 
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V.  Conclusions  And  Recommendations 


Introduction 

The  purpose  of  this  chapter  is  to  discuss  the  conclusions  and  recommendations  of  this 
research  effort.  First,  answers  are  provided  to  the  investigative  questions  which,  when 
taken  as  a  whole,  provide  the  basis  for  answering  the  research  question.  Next,  we 
examine  the  conclusions  and  recommendations  of  this  research  based  on  research  into 
prior  studies  and  results  of  a  mail  survey.  Finally,  recommendations  for  future  research 
are  provided. 

Investigative  Questions  Answered 

This  section  provides  answers  to  investigative  questions  stated  in  Chapter  I  that 
determine  the  most  appropriate  performance  measurement  system  for  Air  Force  vehicle 
maintenance  units. 

Investigative  Question  1.  Haw  are  efficiency,  effectiveness,  and  productivity  related 
to  performance  measurement? 

The  literature  review  examined  the  essential  characteristics  of  a  performance 
measurement  system  and  determined  that  the  important  concepts  of  any  performance 
measurement  system  are  productivity,  efficiency,  and  effectiveness.  Productivity  was 
defined  as  both  a  measure  of  effectiveness  and  efficiency.  Effectiveness  is  achieving  both 
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customer  satisfaction  and  organizational  goals  and  efficiency  is  the  ratio  of  inputs 
consumed  to  outputs  produced. 

Investigative  Question  2.  Why  are  more  relevant  performance  measurement 
indicators  needed  in  government  organizations? 

Examination  of  previous  studies  and  literature  indicated  that  government  organizations 
do  not  utilize  adequate  performance  measurement  systems.  It  was  determined  that 
essential  characteristics  of  an  effective  performance  measurement  system  were  missing  in 
performance  measures  of  government  organizations. 

Investigative  Question  3.  How  do  vehicle  maintenance  officers  perceive  the  VOC 
performance  stcmdard? 

The  literature  review  uncovered  previous  research  by  Weisert  and  Clark  (1972)  and 
Brewer  (1989),  indicating  the  inadequacy  of  current  transportation  performance 
measurements.  Comparison  of  units  using  VOC  measurements  was  not  accurate  and  the 
research  concluded  it  should  not  be  used  to  cross-compare  units.  The  results  of  our 
research  supported  these  conclusions.  Although  vehicle  maintenance  units  were  subject  to 
VOC  standards,  a  large  majority  of  respondents,  88.4  percent,  perceived  the  VOC 
standard  as  inadequate. 

Investigative  Question  4.  Should  independent  factors  be  included  when  establishing  a 
performance  standard? 

This  question  was  answered  based  on  results  of  the  literature  review  and  the  survey 
questionnaire.  The  literature  review  stressed  the  importance  of  including  independent 
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factors  that  impact  performance  when  establishing  a  performance  measurement  system.  In 
addition,  results  of  our  research  indicated  that  nearly  89  percent  of  respondents  believe 
independent  factors  should  be  included  when  establishing  a  performance  standard. 

Investigative  Question  5.  Is  multiple  regression  analysis  suitable  for  evaluating  the 
performance  of  vehicle  maintenance  units? 

The  answer  to  this  question  is  found  in  the  section  entitled  Performance  Measurement 
Systems  for  Vehicle  Maintenance  located  in  chapter  two.  This  section  examined  the 
benefits  of  how  a  regression  model  can  help  evaluate  performance  of  Air  Force  aircraft 
maintenance  units  and  the  U.S.  Postal  Service.  This  section  concluded  that  regression 
models  were  suitable  for  service  organizations  desiring  to  measure  the  effect  of  both 
quantifiable  and  intangible  factors  that  effect  performance  and  cross-compare  units. 
Because  vehicle  maintenance  units  are  a  service  organization  located  in  many  unique 
geographic  areas,  and  experience  many  of  the  same  type  conditions  as  aircraft 
maintenance  units,  regression  analysis  should  be  used  to  evaluate  the  performance. 

Investigative  Question  6.  What  independent  variables  impact  the  performance  of 
vehicle  maintenance  units  the  most? 

Because  the  literature  review  stressed  the  importance  of  limiting  the  number  of  . 
required  performance  measures  to  a  minimum,  the  researchers  decided  to  focus  on 
independent  variables  impacting  vehicle  maintenance  units  the  most.  The  variables 
meeting  this  criteria  were  training  level  of  assigned  personnel,  parts  availability,  available 
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manpower,  budget  available  for  vehicle  maintenance,  tool  and  equipment  availability,  age 
of  vehicle  fleet,  and  experience  (#  of  years)  of  assigned  personnel. 

Investigative  Question  7,  Are  there  differences  in  perceptions,  based  on  rank, 
transportation  e)q>erience,  and  current  position,  about  the  VOC  standard? 

The  mean  response  results  of  this  analysis  can  be  found  in  Tables  5-7.  The  results 
indicated  a  difference  in  perception  based  on  rank,  transportation  experience,  and  current 
position.  However,  Wilcoxon  Rank  Sum  Testing  revealed  that  differences  in  perceptions 
about  a  performance  measurement  standard  occurred  between  the  rank  groups  of  (0-1/0- 
2/and  others)  and  the  rank  group  of  (0-3/0-4/and  0-5).  Additionally,  this  same  procedure 
revealed  that  a  difference  in  perceptions  existed  between  respondents  currently  serving  in 
positions  with  vehicle  maintenance  responsibilities  and  those  respondents  that  did  not 
currently  have  those  same  responsibilities. 

Tnvesrigative  Question  8.  Are  there  differences  in  perceptions,  based  on  rank, 
transportation  experience,  and  current  position,  about  the  most  important  independent 
variables  impacting  vehicle  maintenance  performance? 

The  mean  response  results  for  this  analysis  can  be  found  in  Tables  8-10.  There  appear 
to  be  five  differences  in  perceptions  (Table  1 1).  However,  a  Vifilcoxon  Rank  Sum  Test 
revealed  that  there  were  only  two  statistically  significant  different  mean  responses,  that 
corresponded  to  the  age  of  vehicle  fleet  and  ejq)erience  (#  of  years)  of  assigned  personnel. 
For  the  age  of  the  vehicle  fleet  factor,  the  subgroup  of  0-1/0-2/0-4/0-5  differed  fi'om  the 
respondents  that  were  in  the  0-3/others  subgroup.  Finally,  the  experience  (#  of  years)  of 
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assigned  personnel  factor  differed  significantly  between  the  less  than  one  year  VMO 
experience  subgroup  and  the  all  other  subgroup. 

Conclusions  And  Recommendations 

Many  conclusions  could  be  reached  from  the  results  of  this  research  effort.  The  most 
significant  conclusion  was  that  the  VOC  standard  was  not  viewed  as  an  adequate 
performance  indicator  for  vehicle  m^tenance  units.  Although  this  measure  was  used  to 
compare  vehicle  maintenance  performance  at  different  locations,  the  results  of  this 
research  indicated  that  this  comparison  was  inadequate  because  the  VOC  measure  did  not 
account  for  independent  factors  that  impact  performance.  Additionally,  it  was  determined 
that  the  current  VOC  measure  was  not  established  using  the  concepts  of  productivity, 
effectiveness,  and  efficiency. 

Another  important  conclusion  was  that  of  placing  emphasis  on  a  performance 
measurement  system  that  incorporates  the  use  of  independent  factors.  To  allow  cross- 
comparison  of  vehicle  maintenance  units,  independent  factors  should  be  included  in  the 
performance  measurement  standard  to  account  for  the  specific  operating  conditions  at 
different  locations.  In  keeping  with  the  concept  of  limiting  the  number  of  measures  to  less 
than  15,  this  research  identified  the  seven  most  important  factors  impacting  vehicle 
maintenance  units. 

Another  conclusion  relates  to  the  type  of  performance  measurement  system  compatible 
with  vehicle  maintenance  operations.  Results  indicated  that  the  best  possible  performance 
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measurement  standard  should  employ  multiple  regression  analysis  techniques.  Using  this 
technique,  vehicle  maintenance  units  will  be  able  to  examine  the  effects  of  the  independent 
factors  on  performance  and  use  this  analysis  to  improve  the  decision-making  process.  In 
addition,  VMOs  will  be  able  to  identify  areas  of  strong  and  weak  performance  in  a  timely 
manner  without  having  to  conduct  time-consuming  research. 

The  final  conclusion  was  based  on  perceptions  of  vehicle  maintenance  officers.  VMOs 
and  respondents  with  one  or  greater  years  of  total  time  in  service  or  vehicle  maintenance 
experience  perceived  that  independent  factors  impacted  vehicle  maintenance  performance 
more  than  respondents  who  were  not  VMOs  or  had  little  or  no  experience.  This 
realization  led  to  the  conclusion  that  personnel  with  VMO  responsibilities,  or  a  great 
amount  of  experience,  were  more  knowledgeable  about  the  factors  impacting  vehicle 
maintenance  performance  levels  and,  therefore,  perceived  independent  factors  as  being 
more  important  in  a  performance  measurement  system. 

From  these  conclusions,  a  number  of  recommendations  can  be  made.  First  and 
foremost,  the  current  VOC  standard  should  be  eliminated  and  a  new  standard  established 
which  utilizes  regression  analysis  and  the  seven  independent  factors  identified  in  this 
research. 

If  this  is  not  possible,  and  the  VOC  standard  continues  to  be  utilized,  a  survey  of 
vehicle  maintenance  customers  should  be  performed  to  determine  an  accurate  and 
effective  customer  service  level.  TMs  procedure  would  incorporate  the  concept  of 
effectiveness  into  the  VOC  standard.  To  introduce  the  concept  of  efficiency,  without 
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using  independent  factors  and  regression  analysis  techniques,  however,  would  be  beyond 
the  capabilities  of  current  management  systems.  Although  these  two  recommendations 
would  not  result  in  the  best  solution,  the  new  standard  would  still  be  an  improvement  over 
the  current  VOC  standard. 

Recommendations  For  Future  Research 

Several  aspects  of  this  topic  warrant  fiirther  research.  These  areas  include: 

1 .  Determine  the  optimal  method  of  measuring  the  seven  independent  factors 
identified  in  this  research  so  that  regression  analysis  can  be  performed.  Constraints  to 
consider  when  measuring  these  factors  include  ease  of  measurement,  economic 
considerations,  and  the  use  of  existing  data  retrieval  systems.  This  research  would  involve 
a  study  to  establish  a  regression  analysis  formula  using  the  seven  independent  factors 
identified  in  this  research.  Historical  data  representing  the  factors  could  be  used  to 
validate  the  regression  model. 

2.  Identify  specific  customer  requirements,  varying  by  location,  that  can  also  be 
included  in  the  multiple  regression  formula. 

Based  on  responses  to  question  28,  additional  research  could  be  conducted  that 
includes  the  following: 

1.  Determine  the  strength  of  the  VCO  program,  availability  of  commercial  services  and 
vendors,  and  whether  or  not  fleet  composition  is  considered  important  enough  to  include 
in  a  regression  model. 
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2.  Evaluate  the  “percent  of  vehicles  returned  from  maintenance  within  eight  hours” 
performance  standard  to  determine  if  this  standard  is  also  suitable  for  evaluating  vehicle 
maintenance  performance. 

3.  Conduct  research  into  the  implications  of  adopting  a  performance  standard  that  is 
based  on  vehicle  mission  essential  levels. 

Summary 

This  research  examined  the  adequacy  of  the  VOC  standard  used  to  evaluate  vehicle 
maintenance  performance  throughout  the  Air  Force.  We  reviewed  literature  on  the 
concepts  of  a  good  performance  measurement  system  and  related  these  concepts  to  the 
VOC  standard.  Through  the  use  of  a  mail  survey,  the  inadequacy  of  the  current  VOC 
standard  and  the  importance  of  including  independent  factors  in  a  vehicle  maintenance 
performance  standard  were  identified.  By  using  mean  response  rates  and  Wilcoxon  Rank 
Sum  Testing,  seven  independent  factors  were  identified  that  impact  vehicle  maintenance 
the  most.  Based  on  these  results,  conclusions,  recommendations,  and  areas  for  future 
research  were  presented. 
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Appendix  A:  Operating  EfiBciency  Formula 


OPERATING  EFFICIENCY  =  al*Fl  +  a2*F2  +  a3*F3  + 

a4*F4  +a5*Fi ...  +  al2*(Fl*F2)  + ... 


where  Fi  are  factors  that  influence  operating  efficiency,  and  ai  are  coefficients 
that  quantify  effects  of  changes  in  these  factors. 

The  equation  and  factors  used  to  measure  the  level  of  operating  efficiency  at  mail 
processing  facilities  is  shown  below. 

OE  =  plPVOL  +  P2PVOL2  +  pSSBVOL  +  P4DVOL  +  pSITVOL  +  P6HR  + 
p7HR2  +  P8HR2*VOL  +  p9PS  +  piOPS2  +  pi  lYR  +  pi2YR*MPF  + 
P13PA2*LV0L  +  pl4PA*LVOL*MPF  +  pi5PA*VOLFT  +  pl6PA2*FR  + 
P17PA2*FR2  +  P18V0L*FR  +  P19NSUB  +  p20DELS  +  p21RTS2  + 
p22NSUB*DVOL  +  p23RTS*DVOL  +  p24RTS*NSUB  +  P25RTS*DELS  + 
P26DELS*DVOL, 
where: 

PVOL  =  the  volume  of  mail  processed; 

SBVOL  =  the  volume  of  mail  processed  at  stations  and  branches; 

DVO  =  the  volume  of  mail  delivered; 

IT  VO  =  the  volume  of  incoming  tertiary  mail  processed; 

HR  =  the  proportion  oflabor  in  human  resource  fimctions; 

PS  =  the  square  footage  of  mail  processing  space; 

YR  =  the  age  of  the  facility; 

MPF  =  the  floors  of  the  facility  dedicated  to  mail  processing; 

PA  =  the  percentage  ofletter  volume  processed  on  automated  equipment; 
LVOL  =  the  volume  of  letters  processed; 

VOLFT  =  the  volume  of  mail  processed  per  square  foot  of  space; 

FR  =  the  proportion  of  workforce  made  up  of  flexible  (part-time)  labor; 

NSUB  =  the  number  of  sub-locations  for  ancillary  processing  and  delivery; 

DELS  =  the  number  of  delivery  points; 

RTS  =  the  number  ofletter  carrier  routes 
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Appendix  B:  Pilot  Survey 


Vehicle  Maintenance  Performance  Measurement  Survey 

(Pilot  Study) 

This  survey  is  designed  to  gather  data  based  on  your  opinions  and  experiences  within  the 
vehicle  maintenance  field.  Please  read  each  question  carefully  and  follow  the  instructions 
provided  above  each  set  of  questions.  Use  a  soft  lead  pencil  (#2)  to  mark  your  responses 
on  the  attached  scan  sheet.  If  you  answer  any  of  the  questions  with  an  “Other”  response, 
please  fill  in  the  appropriate  response  in  the  space  provided  on  the  questionnaire.  When 
you  have  completed  the  survey,  please  enclose  the  questionnaire  and  survey  scan  sheet  in 
the  enclosed  pre-addressed  envelope  and  put  it  in  the  mail.  If  possible,  please  respond 
within  one  week  of  receiving  this  survey.  Thank  you  for  your  cooperation  and  assistance. 

Back2round  Information 

Please  circle  the  appropriate  response.  If  you  answer  any  of  the  questions  with  an 
“Other"  response,  please  fill  in  the  appropriate  information  in  the  space  provided. 

1.  What  is  your  present  rank? 

1.  0-1  thru  0-2  2.  0-3  3.  0-4  4.  0-5  5.  0-6  6.  Other _ 


2.  What  is  your  primary  AFSC? 

1.  24T1  2.  24T3  3.  24T4  4.  Other 


3.  What  is  your  duty  AFSC? 

1.  24T1  2.  24T3  3.  24T4  4.  Other 


4.  At  what  level  are  you  currently  performing  your  duties? 

1.  BaseAVing-level  2.  Staff-level  3.  Other _ 

5.  What  command  are  you  currently  assigned  to? 

1.  ACC  2.  AMC  3.  AFMC  4.  PACAF  5.  USAFE  6.  Other 


6.  How  much  experience  have  you  had  as  a  vehicle  maintenance  officer? 

1.  None  2.  Less  than  1  year  3.  1-2  years  4.  More  than  2  years 

7.  Are  you  currently  serving  as  a  vehicle  maintenance  officer?  If  you  answer  No  to  this 
question,  go  directly  to  question  14.  (skip  questions  8  through  13) 

1.  Yes  2.  No 

8.  Are  you  subject  to  vehicle  out-of-commission  (VOC)  standards  fi*om  your  MAJCOM? 

1.  Yes  2.  No 
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9.  Are  you  subject  to  VOC  standards  from  your  base  or  unit? 

1.  Yes  2.  No 

10.  If  you  answered  yes  to  question  9,  are  these  standards  different  from  your  MAJCOM 
standard? 

1.  Yes  2.  No 

1 1 .  If  you  are  subject  to  VOC  standards,  what  is  your  current  standard? 

1.  more  than  10%  2.  10%  3.  9%  4.  8%  5.  Less  than  8% 

12.  Are  you  required  to  brief  or  report  your  VOC  results  on  a  regular  basis? 

1.  Yes  2.  No 

13.  If  you  answered  yes  to  question  12,  who  is  briefed  about  your  VOC  rates  (circle  all 
thatcpply) 

1.  MAJCOM  2.  Wing-level  3.  Group-level  4.  Unit-level 

Vehicle  Maintenance  Performance  Measurement  Factors 

14.  When  manpower  levels  are  established,  various  factors  are  used  to  determine  what 
these  levels  should  be.  Do  you  think  these  type  of  factors,  such  as  available 
manpower,  age  of  vehicle  fleet,  budget  levels,  etc.,  should  be  considered  when 
establishing  an  acceptable  vehicle  maintenance  performance  standard? 

1.  Yes  2.  No 

Using  the  scale  below,  please  rate  each  independent  factor  as  you  perceive  the  perfor¬ 
mance  level  of  a  vehicle  maintenance  unit  being  affected  and  circle  your  response. 

Not  Mildly  Somewhat  Substantially  Highly  Extremely 

Affected  Affected  Affected  Affected  Affected  Affected  Affected 


15.  Available  Manpower  1  2  3  4  5  6  7 

16.  Training  level  of  assigned  personnel  1  2  3  4  5  6  7 

17.  Experience  (#  of  years)  of  assigned  personnel  1  2  3  4  5  6  7 

18.  Budget  available  for  vehicle  maintenance  1  2  3  4  5  6  7 

19.  Age  of  vehicle  fleet  1  2  3  4  5  6  7 


79 


Vehicle  Maintenance  Performance  Measurement  Factors  (Continued) 

Using  the  scale  below,  please  rate  each  independent  factor  as  you  perceive  the  perfor¬ 
mance  level  of  a  vehicle  maintenance  unit  being  affected  and  circle  your  response. 


Not  Mildly  Somewhat  Substantially 

Affected  Affected  Affected  Affected  Affected 


Highly  Extremely 
Affected  Affected 


20.  Parts  availability 

21.  Severity  of  climate 

22.  Availability  of  warranty  service 

23.  Age  of  maintenance  facility 

24.  Size  of  maintenance  facility 

25.  Tool  &  equipment  availability 

26.  Forward  operating  location  responsibilities 

27.  Vehicle  utilization  rates 


1  2  3  4  5  6  7 
1  2  3  4  5  6  7 
1  2  3  4  5  6  7 
1  2  3  4  5  6  7 
1  2  3  4  5  6  7 
1  2  3  4  5  6  7 
1  2  3  4  5  6  7 
1  2  3  4  5  6  7 


Identification  Of  Other  Performance  Factors 

28.  Are  there  other  factors  that  should  be  considered  when  determining  performance 
levels  of  vehicle  maintenance  units  that  are  not  listed  above?  Please  respond  below. 
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Appendix  C:  Pilot  Survey  Correlation  Analysis 


13  'VAR'  Variables:  PERFl  PERF2  PERF3  PERF4  PERF5  PERF6 
PERF7  PERF8  PERF9  PERFIO  PERFl  1  PERF12 
PERF13 


Simple  Statistics 


Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

PERFl 

10 

5.70000 

0.94868 

57.00000 

4.00000 

7.00000 

PERF2 

10 

5.30000 

0.94868 

53.00000 

4.00000 

7.00000 

PERF3 

10 

4.60000 

0.96609 

46.00000 

3.00000 

6.00000 

PERF4 

10 

4.70000 

1.05935 

47.00000 

3.00000 

6.00000 

PERF5 

10 

5.20000 

1.31656 

52.00000 

3.00000 

7.00000 

PERF6 

10 

5.10000 

0.99443 

51.00000 

4.00000 

7.00000 

PERF7 

10 

4.60000 

1.17379 

46.00000 

2.00000 

6.00000 

PERF8 

10 

3.70000 

0.94868 

37.00000 

2.00000 

5.00000 

PERF9 

10 

2.70000 

1.05935 

27.00000 

1.00000 

4.00000 

PERFIO 

10 

3.40000 

1.42984 

34.00000 

1.00000 

6.00000 

PERFl  1 

10 

5.60000 

0.84327 

56.00000 

5.00000 

7.00000 

PERF12 

10 

3.90000 

1.72884 

39.00000 

1.00000 

7.00000 

PERF13 

10 

4.20000 

1.98886 

42.00000 

1.00000 

7.00000 

Cronbach  Coefficient  Alpha  for  RAW  variables:  0.845914 

for  STANDARDIZED  variables:  0.845233 


Raw  Variables  Std.  Variables 

Correlation  Correlation 

with  Total _ Alpha _ with  Total _ Alpha 


Deleted 

Variable 

PERFl 

PERF2 

PERF3 

PERF4 

PERF5 

PERF6 

PERF7 

PERF8 

PERF9 

PERFIO 

PERFl  1 

PERF12 

PERF13 


0.000000 

0.632813 

0.372673 

0.482133 

0.533027 

0.585191 

0.451597 

0.664016 

0.582755 

0.869284 

0.204744 

0.481159 

0.700466 


0.860455 

0.829178 

0.842513 

0.836465 

0.832767 

0.831154 

0.838169 

0.827532 

0.830649 

0.805268 

0.849975 

0.840214 

0.821278 


0.074405 

0.587601 

0.406616 

0.470997 

0.590484 

0.587563 

0.465492 

0.658264 

0.545141 

0.826615 

0.224340 

0.428168 

0.655259 


0.861345 

0.828309 

0.840458 

0.836200 

0.828111 

0.828312 

0.836567 

0.823413 

0.831209 

0.811396 

0.852139 

0.839040 

0.823623 
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Pilot  Survey  Correlation  Analysis  (continued') 


Pearson  Correlation  Coefficients  /  Prob  >  1R|  under  Ho:  Rho=0  /  N  -  10 


PERFl 

PERF2 

PERF3 

PERFl 

0.00000 

0.11111 

0.33945 

0.0 

0.7599 

0.3373 

PERF2 

0.11111 

1.00000 

0.26671 

0.7599 

0.0 

0.4563 

PERF3 

0.33945 

0.26671 

1.00000 

0.3373 

0.4563 

0.0 

PERF4 

0.01106 

0.09950 

0.52112 

0.9758 

0.7845 

0.1224 

PERF5 

0.40922 

0.12454 

0.50667 

0.2403 

0.7317 

0.1350 

PERF6 

0.15311 

0.67133 

0.16192 

0.6728 

0.0335 

0.6549 

PERF7 

-0.01996 

0.21952 

0.13718 

0.9564 

0.5423 

0.7055 

PERF8 

0.13580 

0.48148 

0.09699 

0.7084 

0.1588 

0.7898 

PERF9 

-0.21006 

0.21006 

0.62969 

0.5602 

0.5602 

0.0511 

PERFIO 

-0.14744 

0.72083 

0.20913 

0.6844 

0.0187 

0.5620 

PERFl  1 

0.38889 

0.02778 

-0.08183 

0.2667 

0.9393 

0.8222 

PERF12 

-0.29131 

0.42680 

0.10644 

0.4141 

0.2187 

0.7698 

PERF13 

-0.31800 

0.78911 

0.04626 

0.3706 

0.0067 

0.8990 

PERF4 

PERF5 

PERF6 

PERF7 

0.01106 

0.40922 

0.15311 

-0.01996 

0.9758 

0.2403 

0.6728 

0.9564 

0.09950 

0.12454 

0.67133 

0.21952 

0.7845 

0.7317 

0.0335 

0.5423 

0.52112 

0.50667 

0.16192 

0.13718 

0.1224 

0.1350 

0.6549 

0.7055 

1.00000 

0.28680 

0.03164 

-0.10723 

0.0 

0.4217 

0.9309 

0.7681 

0.28680 

1.00000 

0.49223 

0.63272 

0.4217 

0.0 

0.1484 

0.0496 

0.03164 

0.49223 

1.00000 

0.51403 

0.9309 

0.1484 

0.0 

0.1285 

-0.10723 

0.63272 

0.51403 

1.00000 

0.7681 

0.0496 

0.1285 

0.0 

0.45329 

0.32026 

0.38867 

0.47895 

0.1883 

0.3670 

0.2670 

0.1614 

0.70297 

0.60547 

0.34806 

0.25020 

0.0234 

0.0636 

0.3243 

0.4857 

0.45480 

0.30692 

0.51575 

0.50315 

0.1866 

0.3884 

0.1270 

0.1382 

0.09950 

0.18014 

0.18550 

0.38166 

0.7845 

0.6185 

0.6079 

0.2765 

0.52782 

0.10740 

0.13572 

-0.02190 

0.1169 

0.7678 

0.7085 

0.9521 

0.29533 

0.19520 

0.55056 

0.37124 

0.4074 

0.5889 

0.0991 

0.2909 
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Pilot  Survey  Correlation  Analysis  (continued') 


Pearson  Correlation  CoeflBcients  /  Prob  >  |R|  under  Ho:  Rho=0  /  N 


PERF8 

PERFl 

0.13580 

0.7084 

PERF2 

0.48148 

0.1588 

PERF3 

0.09699 

0.7898 

PERF4 

0.45329 

0.1883 

PERF5 

0.32026 

0.3670 

'PERF6 

0.38867 

0.2670 

PERF7 

0.47895 

0.1614 

PERF8 

1.00000 

0.0 

PERF9 

0.23218 

0.5186 

PERFIO 

0.75359 

0.0118 

PERFl  1 

0.38889 

0.2667 

PERF12 

0.25066 

0.4849 

PERF13 

0.62422 

0.0537 

PERF9 

PERFIO 

-0.21006 

-0.14744 

0.5602 

0.6844 

0.21006 

0.72083 

0.5602 

0.0187 

0.62969 

0.20913 

0.0511 

0.5620 

0.70297 

0.45480 

0.0234 

0.1866 

0.60547 

0.30692 

0.0636 

0.3884 

0.34806 

0.51575 

0.3243 

0.1270 

0.25020 

0.50315 

0.4857 

0.1382 

0.23218 

0.75359 

0.5186 

0.0118 

1.00000 

0.45480 

0.0 

0.1866 

0.45480 

1.00000 

0.1866 

0.0 

-0.27364 

0.33174 

0.4443 

0.3490 

0.46715 

0.64726 

0.1734 

0.0431 

0.45354 

0.90647 

0.1880 

0.0003 

PERFl  1 

PERF12 

0.38889 

-0.29131 

0.2667 

0.4141 

0.02778 

0.42680 

0.9393 

0.2187 

-0.08183 

0.10644 

0.8222 

0.7698 

0.09950 

0.52782 

0.7845 

0.1169 

0.18014 

0.10740 

0.6185 

0.7678 

0.18550 

0.13572 

0.6079 

0.7085 

0.38166 

-0.02190 

0.2765 

0.9521 

0.38889 

0.25066 

0.2667 

0.4849 

-0.27364 

0.46715 

0.4443 

0.1734 

0.33174 

0.64726 

0.3490 

0.0431 

1.00000 

0.04573 

0.0 

0.9002 

0.04573 

1.00000 

0.9002 

0.0 

-0.01325 

0.68507 

0.9710 

0.0288 

10 

PERF13 

-0.31800 

0.3706 

0.78911 

0.0067 

0.04626 

0.8990 

0.29533 

0.4074 

0.19520 

0.5889 

0.55056 

0.0991 

0.37124 

0.2909 

0.62422 

0.0537 

0.45354 

0.1880 

0.90647 

0.0003 

-0.01325 

0.9710 

0.68507 

0.0288 

1.00000 

0.0 
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Appendix  D:  Approval  Notification  Letter 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AIR  FORCE  MIUTARY  PERSONNEL  CENTER 
RANDOLPH  AI R  FORCE  BASE  TEXAS 


MEMORANDUM  FOR  AHT/XP  13  APR  1995 

ATTN;  MSHOUTZ 

FROM:  AFMPC/DPMYMS 

550  C  Street  West,  Ste  35 
Randolph  AFBTX  78150-4737 


SUBJECT:  Request  for  Survey  Approval  (Your  FAX,  1 1  Apr  95) 

The  proposed  survey  submitted  on  behalf  of  Captains  Arzberger  and  Audet  is  approved 
contingent  on  making  the  following  changes; 

a.  Reference  item  4.  Change  item  to  read,  “At  what  organizational  level  are  you 
currently  performing  your  duties?” 

b.  Reference  item  5.  Change  item  to  read,  “To  which  command  are  you  assigned?” 

c.  Reference  item  6.  Change  response  option  3  and  4  to  read,  “3 .  1  but  less  than  2 
years,  4.  2  years  or  more.” 

d.  Reference  scale  for  item  14.  Change  to  “Strongly  Disagree,  Disagree,  Slightly 
Disagree,  Neutral,  Slightly  Agree,  Agree,  Strongly  Agree.” 

e.  Reference  items  15-27.  The  item  stem  and  the  scale  do  not  match.  Please  change 
the  stem  to  read,  “To  what  extent  do  you  believe  each  of  the  following  independent 
factors  impact  the  performance  level  of  a  vehicle  maintenance  unit.  Please  circle  your 
response  for  each  factor.”  Change  the  scale  to  read,  “Very  Great  Extent,  Great  Extent, 
Moderate  Extent,  Slight  Extent,  Not  At  All.” 

Please  forward  a  revised  copy  of  this  survey  for  our  files.  A  survey  control  number 
(SCN)  of  SCN  USAF  95-38  is  assigned  and  expires  on  1  Sep  95.  Questions  regarding 
this  action  can  be  addressed  to  me  at  DSN  487-5680. 


original  signed 
CHARLES  H.  HAMILTON 
Chief,  Survey  Branch 
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Appendix  E:  Performance  Measurement  Survey 


FROM:  Lieutenant  Colonel  Floyd  R.  Anible  19  May  1995 

Air  Force  Institute  of  T echnology  (AFIT/LSM) 

Wright-Patterson  AFB,  OH  45433-7765 

SUBJ:  Vehicle  Maintenance  Performance  Measurement  Survey 

TO:  Survey  Participants 

1 .  The  attached  questionnaire  was  prepared  by  a  research  team  at  the  Air  Force  Institute 
of  Technology,  Wright-Patterson  MB  OH  and  is  sponsored  by  the  Pacific  Air  Forces. 

This  survey  is  designed  to  gather  data,  based  on  your  opinions,  perceptions,  and 
experiences  and  those  of  your  fellow  transportation  maintenance  officers,  concerning  the 
current  vehicle  out-of-commission  (VOC)  standard  and  the  importance  of  independent 
factors  that  affect  the  performance  of  vehicle  maintenance  flights. 

2.  The  goal  of  this  study  is  to  evaluate  the  current  VOC  standard  and  recommend  a  new 
performance  measurement  standard  that  is  based  on  efficiency,  effectiveness,  and 
productivity.  The  report  will  be  reviewed  by  managers  at  every  level  of  the  transportation 
and  logistics  command  structure.  Your  individual  responses  will,  of  course,  be  kept 
strictly  confidential  and  will  be  combined  vvdth  others  to  form  the  basis  for  this  report. 
Although  your  participation  in  this  survey  is  voluntary,  we  strongly  urge  you  to  complete 
the  attached  questionndre  and  return  it  as  soon  as  possible.  The  questionnaire  should  only 
take  about  20  minutes.  This  research  is  for  your  benefit,  and  represents  an  opportunity  to 
make  your  voice  heard  in  choosing  appropriate  measures  of  your  performance. 

3 .  Thank  you  for  your  cooperation  and  assistance  in  this  endeavor.  If  you  have  any 
questions  or  recommendations  regarding  this  survey,  please  contact  Captain  Christopher 
Arzberger  at  DSN  785-7777,  ext.  2105  or  Captain  Larry  Audet  at  DSN  785-7777,  ext. 
2107. 


original  signed 

Floyd  R.  Anible,  Lieutenant  Colonel,  USAF 
Deputy  Head,  Department  of  Logistics  Management 
School  of  Systems  and  Logistics 
Air  Force  Institute  of  Technology 
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Vehif  le  Maintenance  Performance  Measurement  Surve 

This  survey  is  designed  to  gather  data  based  on  your  opinions  and  experiences  within  the 
vehicle  maintenance  field.  Please  read  each  question  carefully  and  follow  the  instructions 
provided  above  each  set  of  questions.  Use  a  soft  lead  pencil  (#2)  to  mark  your  response 
on  the  attached  scan  sheet.  If  you  answer  any  of  the  questions  with  an  "Other"  response, 
please  fill  in  the  appropriate  information  in  the  space  provided  on  the  questionnaire.  When 
you  have  completed  the  survey,  please  enclose  the  questionnaire  and  survey  scan  sheet  in 
the  enclosed  pre-addressed  stamped  envelope  and  put  it  in  the  mail.  If  possible,  please 
respond  within  one  week  of  receiving  this  survey.  Thank  you  for  your  cooperation  and 
assistance. 


Background  Information 

Please  mark  your  response  on  the  scan  sheet  provided.  If  you  answer  any  of  the 
questions  with  an  ” Other*'  response,  fill  in  the  appropriate  information  in  the  space 
provided  on  the  questionnaire 

1.  What  is  your  present  rank? 

1.  0-1  thru  0-2  2.  0-3  3.  0-4  4.  0-5  5.  0-6  6.  Other _ 

2.  What  is  your  primary  AFSC? 

1.  24T1  2.  24T3  3.  24T4  4.  Other _ 

3.  What  is  your  duty  AFSC? 

1.  24T1  2.  24T3  3.  24T4  4.  Other _ 

4.  At  what  organizational  level  are  you  currently  performing  your  duties? 

1.  BaseAVing-level  2.  Staff-level  3.  Other _ 

5.  To  which  command  are  you  assigned? 

1.  ACC  2.  AMC  3.  AFMC  4.  PACAF  5.  USAFE  6.  Other _ 

6.  How  much  experience  have  you  had  as  a  vehicle  maintenance  officer? 

1.  None  2.  Less  than  1  year  3.  1  but  less  than  2  years  4.  2  years  or  more 

7.  Are  you  currently  serving  as  a  vehicle  maintenance  officer  or  in  another  position  with 
vehicle  maintenance  responsibilities?  If  you  answer  No  to  this  question,  go  directly  to 
question  14.  (skip  questions  8  through  13) 

1.  Yes  2.  No 


8.  Are  you  subject  to  vehicle  out-of-commission  (VOC)  standards  from  your  MAJCOM? 
1.  Yes  2.  No 
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9.  Are  you  subject  to  VOC  standards  from  your  base  or  unit? 

1.  Yes  2.  No 

10.  If  you  answered  yes  to  question  9,  are  these  standards  different  from  your  MAJCOM 
standard? 

1.  Yes  2.  No 

1 1 .  If  you  are  subject  to  VOC  standards,  what  is  your  current  standard? 

1.  more  than  10%  2.  10%  3.  9%  4.  8%  5.  Other 


12.  Are  you  required  to  brief  or  report  your  VOC  results  on  a  regular  basis? 

1.  Yes  2.  No 

13.  If  you  answered  yes  to  question  12,  who  is  briefed  about  your  VOC  rates?  (mark  all 
that  apply) 

1.  MAJCOM  2.  Wing  Level  3.  Group  Level  4.  Unit  Level 

Vehicle  Maintenance  Performance  Measurement  Factors 


Use  the  scale  shown  below  to  respond  to  question  14. 


Strongly 

Slightly 

Slightly 

Strongly 

Disagree 

Disagree  Disagree  Neutral 

-2  -3  -  4~ 

Agree 
-  5— 

Agree  Agree 

6  -  7 

14.  When  manpower  levels  are  established,  various  factors  are  used  to  determine  what 
these  levels  should  be.  To  what  extent  do  you  agree  or  disagree  that  these  type  of 
factors,  such  as  available  manpower,  age  of  vehicle  fleet,  budget  levels,  etc.,  should 
be  considered  when  establishing  an  acceptable  vehicle  maintenance  performance 
standard? 

1  2  3  4  5  6  7 

To  what  extent  do  you  believe  each  of  the  following  independent factors  impact  the 
performance  level  of  a  vehicle  maintenance  unit  Please  circle  your  response  for  each 
factor. 


Not 

Slight 

Moderate 

Great 

Very  Great 

At  All 

Extent 

Extent 

Extent 

Extent 

^ - 

- ^ - 

- 3 - 

.  ^  4  -  . 

- 5 

15.  Available  manpower  1  2  3  4  5 
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Vehicle  Maintenance  Performance  Measurement  Factors  (Continued) 


To  what  extent  do  you  believe  each  of  the  following  independent  factors  intact  the 
performance  level  of  a  vehicle  maintenance  unit  Please  circle  your  response  for  each 
factor. 

Not  Slight  Moderate  Great  Very  Great 

At  All  Extent  Extent  Extent  Extent 

1 - 2 - 3 - 4 - 5 


1 6 .  Training  level  of  assigned  personnel 

17.  Experience  (#  of  years)  of  assigned  personnel 

1 8 .  Budget  available  for  vehicle  maintenance 

19.  Age  of  vehicle  fleet 

20.  Parts  availability 

2 1 .  Severity  of  climate 

22.  Availability  of  warranty  service 

23 .  Age  of  maintenance  facility 

24.  Size  of  maintenance  facility 

25.  Tool  &  equipment  availability 

26.  Forward  operating  location  responsibilities 

27.  Vehicle  utilization  rates 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


Identification  Of  Other  Performance  Factors 

28.  If  you  believe  there  are  other  factors  that  should  be  considered  when  determining 
performance  levels  of  vehicle  maintenance  units  that  are  not  listed  above,  please  list 
them  below. 
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Appendix  F:  Performance  Measurement  Survey  Data 


555000001001073195001 

555000002001073195001 

555000003001073195001 

555000004001073195001 

555000005001073195001 

555000006001073195001 

555000007001073195001 

555000008001073195001 

555000009001073195001 

555000010001073195001 

555000011001073195001 

555000012001073195001 

555000013001073195001 

555000014001073195001 

555000015001073195001 

555000016001073195001 

555000017001073195001 

555000018001073195001 

555000019001073195001 

555000020001073195001 

555000021001073195001 

555000022001073195001 

555000023001073195001 

555000024001073195001 

555000025001073195001 

555000026001073195001 

555000027001073195001 

555000028001073195001 

555000029001073195001 

555000030001073195001 

555000031001073195001 

555000032001073195001 

555000033001073195001 

555000034001073195001 

555000035001073195001 

555000036001073195001 

555000037001073195001 

555000038001073195001 

555000039001073195001 

555000040001073195001 

555000041001073195001 


5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 
5326  #0001 


34416411251*  54554554234542 
3332442  2  2  64445443333444 
2221232  53534432322414 
2221142  44344343434344 
222231222  75554443334555 
3332142  74444343433443 
3331242  44444442233422 
2221432  64453543333434 
4332642  54435443222434 
3332442  64333433234434 
22226327  75545553344544 
4341642  75445554323434 
3332632  44544332122433 
3221242  75544555433444 
2232332  75555554333545 
332123 111141 *64443333222322 
2222232  24334332111324 
333224222  74533433233324 
332123111221*75555454344554 
3332132  65555554455544 
4331642  74544553233534 
2223632  33454334333344 
43326411  211  74543353222333 
2332212  55555554344535 
3332242  73445353333544 
2322212  74522141112412 
3232212  75555454355525 
2221612  65544343334444 
3223322  54444554343545 
4332642  24444443322334 
3222532  74435454545333 
3222222  54335453234423 
3332642  64535554334444 
2222642  34434343334434 
3332132222  2  444443543323433 
1221 12111 121163434553222334 
434263 122222244444454333432 
3332642  65545444323424 
2221232  74545343344533 
3221232222  64444332333334 
2221412  64444343233433 
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Performance  Measurement  Survey  Data  (continued') 


555000042001073195001  5326  #0001 
555000043001073195001  5326  #0001 
555000044001073195001  5326  #0001 
555000045001073195001  5326  #0001 
555000046001073195001  5326  #0001 
555000047001073195001  5326  #0001 
555000048001073 195001  5326  #0001 
555000049001073195001  5326  #0001 
555000050001073 195001  5326  #0001 
555000051001073195001  5326  #0001 
555000052001073195001  5326  #0001 
555000053001073195001  5326  #0001 
555000054001073195001  5326  #0001 
555000055001073195001  5326  #0001 
555000056001073195001  5326  #0001 
555000057001073195001  5326  #0001 
555000058001073195001  5326  #0001 
555000059001073195001  5326  #0001 
555000060001073195001  5326  #0001 
555000061001073195001  5326  #0001 
555000062001073195001  5326  #0001 
555000063001073195001  5326  #0001 
555000064001073195001  5326  #0001 
555000065001073195001  5326  #0001 
555000066001073195001  5326  #0001 
555000067001073195001  5326  #0001 
555000068001073195001  5326  #0001 
555000069001073195001  5326  #0001 
555000070001073195001  5326  #0001 
555000071001073195001  5326  #0001 
555000072001073195001  5326  #0001 
555000073001073195001  5326  #0001 
555000074001073195001  5326  #0001 
555000075001073 195001  5326  #0001 
555000076001073195001  5326  #0001 
555000077001073195001  5326  #0001 
555000078001073195001  5326  #0001 
555000079001073195001  5326  #0001 
555000080001073195001  5326  #0001 
555000081001073195001  5326  #0001 


2222542  65555554445545 
2221632  64434342232422 
644144122  2  75553345225455 
333121112  5  1464554544233423 
444214122  75454554234533 
3343642  54543354323444 
333261222  75445554222453 
122113111221*75545552333525 
3332642  65434444234534 
3221242  64444554444544 
4331242  74554543234545 
33336411 1221 17534454231341 1 
3332642  75444543233444 
3331622  74545555433435 
3332612  75555543345554 
2233342  64544542133433 
3332642  44555454544545 
2221232  3  24435353323444 
4332632  54444353234434 
4332132  64545343343433 
4332642  74555453344552 
3333642  65545444325445 
322134111211*75555555555555 
333243222  211 654333311 1 1 123 
2221332  65555554334554 
2221232  75445553433545 
222161 111251 *74545454434534 
3332242  65444553335544 
2333222  75515554112544 
2222242  75555554444545 
2232132  75555443334435 
2222222  65433454233445 
2332242  74535553432544 
3332442  75555554444544 
23323326344  *63444443343443 
433254222  52  75535553234434 
644164111151*75545454323432 
3332612  64444443343434 
2223342  54555453324534 
1221 121 1 1221*64525433323332 
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Performance  Measurement  Survey  Data  (continued) 
555000082001073195001  5326  #0001  3221642  65555555244555 


555000083001073195001  5326  #0001 
555000084001073195001  5326  #0001 
555000085001073195001  5326  #0001 
555000086001073195001  5326  #0001 
555000087001073195001  5326  #0001 
555000088001073195001  5326  #0001 
555000089001073195001  5326  #0001 
555000090001073195001  5326  #0001 
555000091001073195001  5326  #0001 
555000092001073195001  5326  #0001 
555000093001073195001  5326  #0001 
555000094001073195001  5326  #0001 
555000095001073195001  5326  #0001 
555000096001073195001  5326  #0001 
555000097001073195001  5326  #0001 
555000098001073195001  5326  #0001 
555000099001073195001  5326  #0001 
555000100001073195001  5326  #0001 
555000101001073195001  5326  #0001 
555000102001073195001  5326  #0001 
555000103001073195001  5326  #0001 
555000104001073195001  5326  #0001 
555000105001073195001  5326  #0001 
555000106001073195001  5326  #0001 
555000107001073195001  5326  #0001 
555000108001073195001  5326  #0001 
(Manually  Input) 

(Manually  Input) 

(Manually  Input) 

(Manually  Input) 

*denotes  missing  data 


3332652  55444444344433 
622144121252  75445552433435 
2221542  64445353333433 
2241232  64453453234343 
2223642  65554455224433 
2223332  64435452223453 
122123111252  65555543333534 
2221242  *75543354322544 
3332612*252  75535454445545 
2221232  64555553434533 
2332642  65434352233324 
666134111221364445343433533 
6444241111  1*75555453444553 
666113111151*75555453423524 
33212421121 1475444453322433 
2221212  65444232223533 
3333612  15555555355434 
2222632  65545553434545 
333254122  64443342122443 
3331112  75555554444545 
3321 14212222245555553343434 
3332642  75555543332353 
2332342  75435353233525 
3321242  65434343324433 
6444241111  1*75555453444553 
666134111221364445343433533 
122113111221*74445443334334 
33242  64544533344434 
12211311 1221*74445444533443 
33242  64544533344434 
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Appendix  G:  Survey  Correlation  Anslvsis 


14 'VAR' Variables:  QUES14  QXJESIS  QUES16  QUES17  QUES18  QUES19 
QUES20  QUES21  QUES22  QUES23  QUES24  QUES25 
QUES26  QUES27 


Variable 

N 

Mean 

Std  Dev 

QUES14 

112 

6.0000 

1.2592 

QUES15 

112 

4.4286 

0.5807 

QUES16 

112 

4.4911 

0.5850 

QUES17 

112 

4.0536 

0.8147 

QUES18 

112 

4.3839 

0.7259 

QUES19 

112 

4.1071 

0.8736 

QUES20 

112 

4.4286 

0.6938 

QUES21 

112 

3.2679 

0.8273 

QUES22 

112 

2.8839 

0.9177 

QUES23 

112 

2.9732 

0.8849 

QUES24 

112 

3.3661 

0.9104 

QUES25 

112 

4.2411 

0.7503 

QUES26 

112 

3.3750 

0.9408 

QUES27 

112 

3.7411 

0.9079 

Sum 

Minimum 

Maximum 

672.0 

1.0000 

7.0000 

496.0 

3.0000 

5.0000 

503.0 

3.0000 

5.0000 

454.0 

1.0000 

5.0000 

491.0 

2.0000 

5.0000 

460.0 

1.0000 

5.0000 

496.0 

3.0000 

5.0000 

366.0 

1.0000 

5.0000 

323.0 

1.0000 

5.0000 

333.0 

1.0000 

5.0000 

377.0 

1.0000 

5.0000 

475.0 

1.0000 

5.0000 

378.0 

1.0000 

5.0000 

419.0 

1.0000 

5.0000 

Cronbach  CoeflBcient  Alpha  for  RAW  variables:  0.842663 

for  STANDARDIZED  variables:  0.855062 


Raw  Variables  Std.  Variables 


Deleted  Correlation  Correlation 


Variable 

QUES14 

QUES15 

QUES16 

QUES17 

QUES18 

QUES19 

QUES20 

QUES21 

QUES22 

QUES23 

QUES24 

QUES25 

QUES26 

QUES27 


with  Total 
0.237827 
0.418987 
0.467920 
0.488335 
0.515865 
0.507053 
0.529522 
0.568915 
0.521115 
0.594637 
0.601292 
0.605330 
0.494689 
0.426081 


Alpha 

0.858387 

0.836837 

0.834719 

0.832112 

0.831021 

0.830843 

0.830614 

0.827098 

0.829889 

0.824995 

0.824369 

0.825783 

0.831764 

0.836266 


with  Total 
0.255550 
0.415107 
0.465638 
0.494091 
0.521240 
0.514479 
0.534215 
0.576533 
0.515606 
0.589491 
0.603885 
0.609376 
0.493101 
0.435555 


Alpha 

0.859444 

0.850548 

0.847659 

0.846017 

0.844440 

0.844834 

0.843683 

0.841197 

0.844768 

0.840431 

0.839577 

0.839251 

0.846074 

0.849383 
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Correlation  Analysis  ('continued') 


Pearson  Correlation  Coefficients  /  Prob  >  |R|  under  Ho;  Rho=0/  N  =  1 12 


QUES14 

QUES14 

1.00000 

0.0 

QUES15 

0.32034 

0.0006 

QUES16 

0.28127 

0.0027 

QUES17 

0.05269 

0.5811 

QUESIS 

0.12813 

0.1782 

QUES19 

0.17198 

0.0698 

QUES20 

0.20625 

0.0291 

QUES21 

0.05189 

0.5869 

QUES22 

0.14032 

0.1400 

QUES23 

0.07277 

0.4458 

QUES24 

0.10217 

0.2838 

QUES25 

0.27652 

0.0032 

QUES26 

0.16730 

0.0779 

QUES27 

0.05516 

0.5635 

QUES15 

QUES16 

0.32034 

0.28127 

0.0006 

0.0027 

1.00000 

0.27656 

0.0 

0.0032 

0.27656 

1.00000 

0.0032 

0.0 

0.23667 

0.34123 

0.0120 

0.0002 

0.24731 

0.27331 

0.0086 

0.0035 

0.28161 

0.28392 

0.0026 

0.0024 

0.32265 

0.23147 

0.0005 

0.0141 

0.26523 

0.32143 

0.0047 

0.0005 

0.09418 

0.17424 

0.3233 

0.0662 

0.11020 

0.26927 

0.2474 

0.0041 

0.29702 

0.26838 

0.0015 

0.0042 

0.29834 

0.38463 

0.0014 

0.0001 

0.19789 

0.23529 

0.0365 

0.0125 

0.28074 

0.25851 

0.0027 

0.0059 

QUES17 

QUES18 

0.05269 

0.12813 

0.5811 

0.1782 

0.23667 

0.24731 

0.0120 

0.0086 

0.34123 

0.27331 

0.0002 

0.0035 

1.00000 

0.19340 

0.0 

0.0410 

0.19340 

1.00000 

0.0410 

0.0 

0.32096 

0.37494 

0.0006 

0.0001  , 

0.21403 

0.40376 

0.0235 

0.0001 

0.35279 

0.26227 

0.0001 

0.0052 

0.33371 

0.50023 

0.0003 

0.0001 

0.46436 

0.38080 

0.0001 

0.0001 

0.41060 

0.31708 

0.0001 

0.0007 

0.34712 

0.42400 

0.0002 

0.0001 

0.36142 

0.24899 

0.0001 

0.0081 

0.17726 

0.24789 

0.0615 

0.0084 

Correlation  Analysis  (continued") 


Pearson  Correlation  CoefiBcients  /  Prob  >  |R|  under  Ho:  Rho-0/  N  —  1 12 


QUES19 

QUES20 

QUES14 

0.17198 

0.0698 

0.20625 

0.0291 

QUESIS 

0.28161 

0.0026 

0.32265 

0.0005 

QUES16 

0.28392 

0.0024 

0.23147 

0.0141 

QUES17 

0.32096 

0.0006 

0.21403 

0.0235 

QUESIS 

0.37494 

0.0001 

0.40376 

0.0001 

QUES19 

1.00000 

0.0 

0.36948 

0.0001 

QUES20 

0.36948 

0.0001 

1.00000 

0.0 

QUES21 

0.38377 

0.0001 

0.44175 

0.0001 

QUES22 

0.25162 

0.0074 

0.30522 

0.0011 

QUES23 

0.35336 

0.0001 

0.26833 

0.0042 

QUES24 

0.27875 

0.0029 

0.29139 

0.0018 

QUES25 

0.31758 

0.0006 

0.47469 

0.0001 

QUES26 

0.25759 

0.0061 

0.33125 

0.0004 

QUES27 

0.30790 

0.0010 

0.23498 

0.0126 

QUES21 

QUES22 

QUES23 

0.05189 

0.14032 

0.07277 

0.5869 

0.1400 

0.4458 

0.26523 

0.09418 

0.11020 

0.0047 

0.3233 

0.2474 

0.32143 

0.17424 

0.26927 

0.0005 

0.0662 

0.0041 

0.35279 

0.33371 

0.46436 

0.0001 

0.0003 

0.0001 

0.26227 

0.50023 

0.38080 

0.0052 

0.0001 

0.0001 

0.38377 

0.25162 

0.35336 

0.0001 

0.0074 

0.0001 

0.44175 

0.30522 

0.26833 

0.0001 

0.0011 

0.0042 

1.00000 

0.37357 

0.37909 

0.0 

0.0001 

0.0001 

0.37357 

1.00000 

0.49533 

0.0001 

0.0 

0.0001 

0.37909 

0.49533 

1.00000 

0.0001 

0.0001 

0.0 

0.47871 

0.41794 

0.63854 

0.0001 

0.0001 

0.0001 

0.31594 

0.32883 

0.40330 

0.0007 

0.0004 

0.0001 

0.42539 

0.29085 

0.33681 

0.0001 

0.0019 

0.0003 

0.34507 

0.25554 

0.31649 

0.0002 

0.0065 

0.0007 
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Correlation  Analysis  (continued') 


Pearson  Correlation  Coefficients  /  Prob  >  |R|  under  Ho:  Rho=0/  N  =  1 12 


QUES14 

QUES15 

QUES16 

QUES17 

QUES18 

QUES19 

QUES20 

QUES21 

QUES22 

QUES23 

QUES24 

QUES25 

QUES26 

QUES27 


QUES24 

0.10217 

0.2838 

0.29702 

0.0015 

0.26838 

0.0042 

0.41060 

0.0001 

0.31708 

0.0007 

0.27875 

0.0029 

0.29139 

0.0018 

0.47871 

0.0001 

0.41794 

0.0001 

0.63854 

0.0001 

1.00000 

0.0 

0.42359 

0.0001 

0.34317 

0.0002 

0.32282 

0.0005 


QUES25 

0.27652 

0.0032 

0.29834 

0.0014 

0.38463 

0.0001 

0.34712 

0.0002 

0.42400 

0.0001 

0.31758 

0.0006 

0.47469 

0.0001 

0.31594 

0.0007 

0.32883 

0.0004 

0.40330 

0.0001 

0.42359 

0.0001 

1.00000 

0.0 

0.33022 

0.0004 

0.30406 

0.0011 


QUES26 

0.16730 

0.0779 

0.19789 

0.0365 

0.23529 

0.0125 

0.36142 

0.0001 

0.24899 

0.0081 

0.25759 

0.0061 

0.33125 

0.0004 

0.42539 

0.0001 

0.29085 

0.0019 

0.33681 

0.0003 

0.34317 

0.0002 

0.33022 

0.0004 

1.00000 

0.0 

0.27291 

0.0036 


QUES27 

0.05516 

0.5635 

0.28074 

0.0027 

0.25851 

0.0059 

0.17726 

0.0615 

0.24789 

0.0084 

0.30790 

0.0010 

0.23498 

0.0126 

0.34507 

0.0002 

0.25554 

0.0065 

0.31649 

0.0007 

0.32282 

0.0005 

0.30406 

0.0011 

0.27291 

0.0036 

1.00000 

0.0 
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Acronym 

Question 

Acronym 

Question 

Acronym 

Question 

VMOEXP 

6 

VMNOW 

7 

TRNG 

16 

MAJVOC 

8 

BASVOC 

9 

EXPER 

17 

DIFVOC 

10 

CMND 

5 

BDGT 

18 

RTE 

11 

BRFVOC 

12 

AGE 

19 

WHOBRF 

13 

FCTR14 

14 

PRTS 

20 

MNPWR 

15 

DAFS 

3 

LVL 

4 

CLMT 

21 

WRNTY 

22 

FCAGE 

23 

FCSZ 

24 

TL 

25 

PAFS 

2 

FOL 

26 

UTL 

27 

RNK 

1 

(STATISTK  4.0  III  THESIS,  08/13/95, 22:12) 

VMOEXP  VMNOW 

MAJVOC 

BASVOC 

DIFVOC 

CMND 

N 

112 

112 

36 

36 

27 

112 

SUM 

354 

201 

59 

54 

56 

397 

MEAN 

3.1607 

1.7946 

1.6389 

1.5000 

2.0741 

3.5446 

SD 

1.0701 

0.4677 

1.2907 

0.5606 

1.0350 

1.9447 

MINIMUM 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

MEDIAN 

3.0000 

2.0000 

1.0000 

1.0000 

2.0000 

3.0000 

MAXIMUM 

5.0000 

4.0000 

7.0000 

3.0000 

5.0000 

6.0000 

KURTOSIS 

-0.2791 

3.4187 

9.6195 

-0.8430 

2.8588 

-1.6030 

RTE 

BRFVOC 

WHOBRF 

FCTR14 

MNPWR 

DAFS 

N 

26 

28 

12 

111 

112 

112 

SUM 

72 

39 

28 

665 

498 

305 

MEAN 

2.7692 

1.3929 

2.3333 

5.9910 

4.4464 

2.7232 

SD 

1.4229 

0.8317 

1.2309 

1.2613 

0.6273 

0.8405 

MINIMUM 

1.0000 

1.0000 

1.0000 

1.0000 

3.0000 

2.0000 

MEDIAN 

2.0000 

1.0000 

2.0000 

6.0000 

4.0000 

3.0000 

MAXIMUM 

5.0000 

5.0000 

4.0000 

7.0000 

7.0000 

6.0000 

KURTOSIS 

-1.0974 

10.702 

-1.4352 

3.1949 

1.1936 

4.0375 

TRNG 

EXPER 

BDGT 

AGE 

PRTS 

LVL  •  1 

N 

112 

112 

112 

112 

112 

112  I 

SUM 

503 

456 

489 

460 

496 

198  1 

MEAN 

4.4911 

4.0714 

4.3661 

4.1071 

4.4286 

1.7679 

SD 

0.5850 

0.8133 

0.7352 

0.8736 

0.6938 

0.7825 

MINIMUM 

3.0000 

1.0000 

2.0000 

1.0000 

3.0000 

1.0000 

MEDIAN 

5.0000 

4.0000 

5.0000 

4.0000 

5.0000 

2.0000 

MAXIMUM 

5.0000 

5.0000 

5.0000 

5.0000 

5.0000 

4.0000 

KURTOSIS 

-0.5577 

0.7444 

-0.1986 

-0.1348 

-0.5702 

0.4696 
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CLMT 

WRNTY 

FCAGE 

FCSZ 

3L 

PAFS 

N 

112 

112 

112 

112 

112 

112 

SUM 

368 

324 

332 

376 

475 

305 

MEAN 

3.2857 

2.8929 

2.9643 

3.3571 

4.2411 

2.7232 

SD 

0.8429 

0.9139 

0.8897 

0.9090 

0.7503 

0.8077 

MINIMUM 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

2.0000 

MEDIAN 

3.0000 

3.0000 

3.0000 

3.0000 

4.0000 

3.0000 

MAXIMUM 

5.0000 

5.0000 

5.0000 

5.0000 

5.0000 

6.0000 

KURTOSIS 

0.0825 

-0.1391 

-0.1670 

-0.2921 

1.6184 

5.0649 

FOL 

UTL 

RNK 

N 

112 

112 

112 

SUM 

379 

418 

323 

MEAN 

3.3839 

3.7321 

2.8839 

SD 

0.9420 

0.9102 

1.1527 

MINIMUM 

1.0000 

1.0000 

1.0000 

MEDIAN 

3.0000 

4.0000 

3.0000 

MAXIMUM 

5.0000 

5.0000 

6.0000 

KURTOSIS 

-0.1917 

-0.2772 

1.5831 

BREAKDOWN  FOR  FCTR14 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D.  1 

RNK 

1 

6 

39 

6.5000 

0.5477  1 

RNK 

2 

38 

222 

5.8421 

1.3661 

RNK 

3 

48 

286 

5.9583 

RNK 

4 

12 

71 

5.9167 

1.6214 

RNK 

6 

8 

54 

6.7500 

0.4629 

OVERALL 

112 

672 

6.0000 

1.2592 

CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  FCTR14 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

VMOEXP 

1 

15 

92 

6.1333 

1.5523 

VMOEXP 

2 

9 

54 

6.0000 

0.7071 

VMOEXP 

3 

33 

190 

5.7576 

1.4149 

VMOEXP 

4 

53 

324 

6.1132 

1.1546 

VMOEXP 

5 

2 

12 

6.0000 

1.4142 

OVERALL 

112 

672 

6.0000 

1.2592 

CASES  INCLUDED  112  MISSING  CASES  0 
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L 


BREAKDOWN  FOR  FCTR14 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D.  B 

VMNOW 

1 

25 

162 

6.4800 

VMNOW 

2 

87 

510 

5.8621 

1.3397 

OVERALL 

112 

672 

6.0000 

1.2592 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  TRNG 

VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

RNK 

1 

6 

27 

4.5000 

0.5477 

RNK 

2 

38 

171 

4.5000 

RNK 

3 

48 

214 

4.4583 

0.6174 

RNK 

4 

12 

54 

4.5000 

0.5222 

RNK 

6 

8 

37 

4.6250 

0.5175 

OVERALL 

112 

503 

4.4911 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  PRTS 

[variable 

LEVEL 

N 

SUM 

MEAN 

S.D.  1 

RNK 

1 

6 

25 

4.1667 

RNK 

2 

38 

170 

4.4737 

RNK 

3 

48 

208 

4.3333 

RNK 

4 

12 

56 

4.6667 

RNK 

6 

8 

37 

4.6250 

0.5175 

OVERALL 

112 

496 

4.4286 

0.6938 

CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  MNPWR 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

RNK 

1 

6 

25 

4.1667 

0.7528 

RNK 

2 

38 

167 

4.3947 

0.6384 

RNK 

3 

48 

215 

4.4792 

0.5454 

RNK 

4 

12 

51 

0.4423 

RNK 

6 

8 

38 

0.4629 

OVERALL 

112 

496 

4.4286 

0.5827 

CASES  INCLUDED  112  MISSING  CASES  0 


BREAKDOWN  FOR  BDGT 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

.S.D,  1 

RNK 

1 

6 

29 

4.8333 

0.4082  1 

RNK 

2 

38 

165 

4.3421 

RNK 

3 

48 

207 

4.3125 

0.7192 

RNK 

4 

12 

50 

4.1667 

0.7177 

RNK 

6 

8 

38 

0.7071 

OVERALL 

112 

489 

4.3661 

0.7352 

CASES  INCLUDED  1 12  MISSING  CASES  0 
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BREAKDOWN  FOR  TL 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

RNK 

1 

6 

23 

3.8333 

0.9832 

RNK 

2 

38 

165 

4.3421 

RNK 

3 

48 

200 

4.1667 

RNK 

4 

12 

50 

4.1667 

0.7177 

RNK 

6 

8 

37 

4.6250 

0.5175 

OVERALL 

112 

475 

4.2411 

CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  AGE 


CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  EXPER 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

RNK 

1 

6 

22 

3.6667 

1.0328 

RNK 

2 

38 

150 

3.9474 

0.9850 

RNK 

3 

48 

197 

4.1042 

0.6916 

RNK 

4 

12 

51 

4.2500 

0.6216 

RNK 

6 

8 

36 

4.5000 

0.5345 

OVERALL 

112 

456 

4.0714 

0.8133 

CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  TRNG 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D.  1 

VMNOW 

1 

25 

113 

4.5200 

VMNOW 

2 

87 

390 

4.4828 

OVERALL 

112 

503 

4.4911 

CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  PRTS 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D.  1 

VMNOW 

1 

25 

112 

VMNOW 

2 

87 

384 

4.4138 

OVERALL 

112 

496 

4.4286 

CASES  INCLUDED  1 12  MISSING  CASES  0 
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BREAKDOWN  FOR  MNPWR 


VARIABLE  LEVEL 

N 

SUM 

MEAN 

S.D. 

VMNOW  1 

25 

111 

4.4400 

VMNOW  2 

87 

385 

4.4253 

OVERALL 

112 

496 

4.4286 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  BDGT 

VARIABLE  LEVEL 

SUM 

MEAN 

S.D. 

VMNOW  1 

25 

111 

4.4400 

IQ  SI 

VMNOW  2 

87 

378 

4.3448 

OVERALL 

112 

489 

4.3661 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  TL 

[variable  level 

N 

SUM 

MEAN 

S.D. 

1  VMNOW  1 

25 

107 

j^Qjr 

1  VMNOW  2 

87 

368 

4.2299 

1  OVERALL 

112 

475 

4.2411 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  AGE 

VARIABLE  LEVEL 

N 

SUM 

MEAN 

S.D. 

VMNOW  1 

25 

103 

VMNOW  2 

87 

357 

1  OVERALL 

112 

460 

mmuStm 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  EXPER 

VARIABLE  LEVEL 

N 

SUM 

MEAN 

S.D. 

VMNOW  1 

25 

107 

VMNOW  2 

87 

349 

4.0115 

OVERALL 

112 

456 

QOS 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  IRNG 

VARIABLE  LEVEL 

N 

SUM 

MEAN 

S.D. 

VMOEXP  1 

15 

71 

4.7333 

0.457 

VMOEXP  2 

9 

40 

4.4444 

0.726 

VMOEXP  3 

35 

155 

4.4286 

0.557 

VMOEXP  4 

53 

237 

4.4717 

OVERALL 

112 

503 

4.4911 

0.585 

CASES  INCLUDED  1 12  MISSING  CASES  0 
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£ 


BREAKDOWN  FOR  PRTS 


VARIABLE 

LEVEL 

N 

MEAN 

S.D. 

VMOEXP 

1 

15 

66 

4.4000 

0.6325 

VMOEXP 

2 

9 

39 

4.3333 

1.0000 

VMOEXP 

3 

35 

150 

4.2857 

0.7886 

VMOEXP 

4 

53 

241 

4.5472 

0.5740 

OVERALL 

112 

496 

4.4286 

0.6938 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  MNPWR 

VARIABLE 

LEVEL 

N 

MEAN 

S.D.  II 

WOEXP 

1 

15 

70 

4.6667 

VMOEXP 

2 

9 

37 

4.1111 

VMOEXP 

3 

35 

151 

4.3143 

VMOEXP 

4 

53 

238 

4.4906 

0.5415 

OVERALL 

112 

498 

4.4464 

0.6273 

CASES  INCLUDED  1 12  MISSING  CASES  0 


BREAKDOWN  FOR  BDGT 


VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

VMOEXP 

1 

15 

66 

4.4000 

0.8281 

VMOEXP 

2 

9 

39 

4.3333 

VMOEXP 

3 

35 

153 

4.3714 

VMOEXP 

4 

53 

231 

4.3585 

OVERALL 

112 

489 

4.3661 

0.7352  1 

CASES  INCLUDED  112 

MISSING  CASES  0 

BREAKDOWN  FOR  TL 

VARIABLE 

LEVEL 

N 

SUM 

MEAN 

S.D. 

VMOEXP 

1 

15 

68 

4.5333 

0.5164 

VMOEXP 

2 

9 

36 

4.0000 

VMOEXP 

3 

35 

142 

4.0571 

VMOEXP 

4 

53 

229 

4.3208 

0.6729 

OVERALL 

112 

475 

4.2411 

CASES  INCLUDED  112  MISSING  CASES  0 


BREAKDOWN  FOR  AGE 


VARIABLE 

LEVEL 

SUM 

MEAN 

S.D. 

VMOEXP 

1 

15 

59 

3.9333 

1.2228 

VMOEXP 

2 

9 

42 

4.6667 

VMOEXP 

3 

35 

141 

4.0286 

0.8220 

VMOEXP 

4 

53 

218 

4.1132 

0.8242 

OVERALL 

112 

460 

4.1071 

0.8736 

CASES  INCLUDED  1 12  MISSING  CASES  0 
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BREAKDOWN  FOR  EXPER 


LEVEL 

N 

MEAN 

S.D. 

VMOEXP 

1 

15 

64 

4.2667 

; 

VMOEXP 

2 

9 

28 

3.1111 

■ES 1 

VMOEXP 

3 

35 

145 

4.1429 

VMOEXP 

4 

53 

219 

4.1321 

, 

OVERALL 

112 

456 

4.0714 

CASES  INCLUDED  112  MISSING  CASES  0 


RANK  SUM  TWO-SAMPLE  TEST  FOR  AGE  BY  03/OTHER 


03/OTHER 

RANK  SUM 

SAMPLE  SIZE 

USTAT 

MEAN  RANK 

1 

2213.00 

46 

1132.00 

48.1 

2 

4115.00 

66 

1904.00 

62.3 

TOTAL 

6328.00 

112 

NORMAL  APPROXIMATION  WTIH  CONTINUITY  CORRECTION  (z-statistic):  2.280 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION:  0.0226 


RANK  SUM  TWO-SAMPLE  TEST  FOR  AGE  BY  ZERO  YEARS  EXPERIENCE 


ZERO  YEARS 

RANK  SUM 

SAMPLE  SIZE 

USTAT 

MEAN  RANK 

1 

911.00 

16 

775.00 

56.9 

2 

5417.00 

96 

761.00 

56.4 

TOTAL 

6328.0 

112 

NORMAL  APPROXIMATION  WITH  CONTINUnY  CORRECTION  (z-statistic):  0.054 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION: 


RANK  SUM  TWO-SAMPLE  TEST  FOR  EXPERIENCE  BY  LESS  THAN  ONE  YEAR  EXPERIENCE 

T  .F.SS  ONF.  RANK  SUM  SAMPLE  SIZE _ U  STAT - MEAN  RANK 

1  247.50  9  202.50  27.5 

2  6080.50  103  724.50  59.0 

TOTAL  6328.00  112 


NORMAL  APPROXIMATION  WITH  CONTINUITY  CORRECTION  (z-statistic):  2.788 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION: 


RANK  SUM  TWO-SAMPLE  TEST  FOR  TOOLS  BY  0-1  THRU  0-2  | 

LESS  ONE 

RANK  SUM 

SAMPLE  SIZE 

USTAT 

MEAN  RANK 

1 

249.00 

6 

228.00 

41.5 

2 

6079.00 

106 

408.00 

57.3 

TOTAL 

6328.00 

112 

NORMAL  APPROXIMATION  WITH  CONTINUnY  CORRECTION  (z-statistic):  1. 157  | 

11  TWO-TABLED  P-VALUE  FOR  NORMAL  APPROXIMATION: 

0.2475  1 
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RANK  SUM  TWO-SAMPLE  TEST  FOR  EXPERIENCE  BY  01/02/03 


01/02/03 


TOTAL 


RANK  SUM 


2291.00 

4037.00 

6328.00 


SAMPLE  SIZE 


44 

68 

112 


USTAT 


1301.00 

1691.00 


MEAN  RANK 


52.1 

59.4 


NORMAL  APPROXIMATION  WITH  CONTINUITY  CORRECTION  (z-statistic):  1. 159 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION;  0.2466 


RANK  SUM  TWO-SAMPLE  TEST  FOR  QUESTION  14  BY  1-LESS  THAN  TWO 


RANK  SUM 

SAMPLE  SIZE 

USTAT 

MEAN  RANK 

1788.00 

35 

1158.00 

51.1 

4540.00 

77 

1537.00 

59.0 

6328.00 

112 

TOTAL 


NORMAL  APPROXIMATION  WITH  COMnNUTTY  CORRECTION  (z-statistic);  1.186 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION:  0.2355 


RANK  SUM  TWO-SAMPLE  TEST  FOR  QUESTION  14  BY  VMONOW 


VMONOW 


1 

2 

TOTAL 


NORMAL  APPROXIMATION  WITH  CONTINUnY  CORRECTION  (z-statistic):  2. 142 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION:  0.0322 


RANK  SUM 

SAMPLE  SIZE 

USTAT 

MEAN  RANK 

1719.5 

25 

1394.50 

68.8 

4608.5 

87 

780.50 

53.0 

6328.0 

112 

RANK  SUM  TWO-SAMPLE  TEST  FOR  QUESTION  14  BY  0-1/0-2/OTHER 


0-1/0-2/OTHER  RANK  SUM  SAMPLE  SIZE 


1  1023.5  14 

2  5304.5  98 

TOTAL  6328.0  112 


USTAT 


918.50 

453.50 


MEAN  RANK 


73.1 

54.1 


NORMAL  APPROXIMATION  WITH  CONUNUITY  CORRECTION  (z-statistic):  2.041 
TWO-TAILED  P-VALUE  FOR  NORMAL  APPROXIMATION:  0.0412 


Appendix  I:  Demographic  Data 


1.  What  is  your  present  rank? 


0-1  thru  0-2 

6 

5.4% 

0-3 

38 

33.9% 

0-4 

48 

42.9% 

0-5 

12 

10.7% 

0-6 

0 

0% 

Other 

8 

7.1% 

2.  What  is  your  primary  AFSC? 

24T1 

0 

0% 

24T3 

47 

42% 

24T4 

56 

50% 

Other 

9 

8% 

3.  What  is  your  duty  AFSC? 

24T1 

0 

0% 

24T3 

50 

44.6% 

24T4 

49 

43.8% 

Other 

13 

11.6% 

4.  At  what  organizational 

level  are  you  currently  performing  your  duties? 

BaseAVing-Level 

46 

41.1% 

StafiF-Level 

51 

45.5% 

Other 

15 

13.4% 

5.  To  which  command  are  you  assigned? 


ACC 

15 

13.4% 

AMC 

35 

31.3% 

AFMC 

12 

10.7% 

PACAF 

8 

7.1% 

USAFE 

5 

4.5% 

Other 

37 

33.0% 
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6.  How  much  experience  have  you  had  as  a  vehicle  maintenance  officer? 


None 

15 

13.4% 

Less  than  1  year 

7 

6.3% 

1  but  less  than  2  years 

35 

31.3% 

2  years  or  more 

55 

49.1% 

7.  Are  you  currently  serving  as  a  vehicle  maintenance  officer  or  in  another  position 
with  vehicle  maintenance  responsibilities? 

Yes  27  24.1% 

No  85  75.9% 

8.  Are  you  subject  to  vehicle  out-of-commission  (VOC)  standards  from  your 
MAJCOM? 

Yes  23  63.9% 

No  13  36.1% 

9.  Are  you  subject  to  VOC  standards  from  your  base  or  unit? 

Yes  20  55.6% 

No  16  44.4% 

10.  If  you  answered  yes  to  question  9,  are  these  standards  different  from  your 
MAJCOM  standard? 

Yes  7  26.9% 

No  19  73.1% 

11.  If  you  are  subject  to  VOC  standards,  what  is  your  current  standard? 


More  than  10% 

3 

11.1% 

10% 

14 

51.9% 

9% 

1 

3.7% 

8% 

2 

7.4% 

Other 

7 

25.9% 

12.  Are  you  required  to  brief  or  report  your  VOC  results  on  a  regular  basis? 

Yes  20  71.4% 

No  8  28.6% 
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13.  If  you  answered  yes  to  question  12,  who  is  briefed  about  your  VOC  rates  (mark 
all  that  apply)! 

MAJCOM  14  77.7% 

Wing  Level  12  66.6% 

Group  Level  11  61.1% 

Unit  Level  14  77.7% 

(10/18  respondents  briefed  all  4  levels/55.5%) 
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